Core responsibilities
- Ingest business data and events
- Productionise transformations and recovery
- Put in place quality and freshness checks
Technical skills
- SQL, Spark or stream processing
- Partitioning, schemas and idempotency
- Orchestration and data contracts
Behavioural skills
- Coordinate producers and consumers
- Balance quality, latency and compute cost
Typical deliverable
A rerunnable pipeline that handles late events without duplicating data.
Possible assessment methods
Design incident recovery for out-of-order events and justify the quality checks.
