What is Hotrod?

Hotrod is a general-purpose data processing platform, built to simplify data engineering workloads - the ingestion, transformation and movement of data from a sources to a destinations.

As a general-purpose platform for data engineering, Hotrod provides an integrated solution for:

1. Defining data processing workloads in a consistent, repeatable manner, using Hotrod's Pipe Language
2. Running those workloads (called Pipes), using Hotrod Agents
3. Scaling workloads across multiple servers, whether on the cloud, on-premise, or both
4. Managing and monitoring running Pipes, by collecting and exposing metrics on their execution

Hotrod brings some specific advantages to data processing problems:

1. All data processing workloads are defined in YAML, using Hotrod's Pipe Language. With a shallow learning curve, setting up new data processing tasks becomes much faster to implement.
2. Hotrod's architecture makes it easy to scale up and distribute data processing workloads, while keeping workloads centrally managed.
3. In terms of security and privacy, Hotrod is a self-hosted and managed solution, which provides complete control over where and how your data is moved.

Hotrod supports a variety of data sources and destinations which are common to modern data engineering use cases. While Hotrod is primarily designed to handle streaming data and continuous (or near-continuous) workloads, it can handle most ETL and batch workloads as well.