Senior Data Engineer - Real Time Streams

You will be a senior engineer on the team that owns the real-time data platform. The platform turns operational events (orders, driver locations, geofence transitions, shifts, MQTT session events) into live analytics, routing inputs, and warehoused data across Postgres/TimescaleDB, ClickHouse, and BigQuery.

You will own the platform end-to-end: design, implementation, deployment, and production operation of the streaming jobs, CDC pipelines, Kafka Connect bridges, and downstream sinks that move this data.

Responsibilities

Designing, implementing, and operating stateful Apache Flink streaming pipelines: keyed state, windowing, watermarks, timers, side outputs, custom sources/sinks.

Designing, developing, deploying, and maintaining Change Data Capture (CDC) pipelines using Flink CDC or Debezium, moving operational database changes into the streaming platform with correct snapshot/incremental handling, schema evolution, and downstream idempotency.

Building and operating Kafka and Kafka Connect pipelines: topic and partition design, source/sink connectors in distributed mode, schema and converter management, and deadletter routing.

Owning the path from Kafka → Flink → Postgres / TimescaleDB / ClickHouse / BigQuery, including schema design, idempotency strategy, batch tuning, and observability. Diagnosing and fixing production issues: checkpoint failures, backpressure, state growth, sink slowness, autoscaler oscillation, restart loops.

Hardening the platform: delivery guarantees, watermarks, DLQ handling, schema migrations, alerting coverage. * Code reviews and mentoring mid-level engineers. Contributing to the deployment side: Helm charts, ArgoCD applications, GKE configuration, Grafana dashboards, Prometheus alert rules.

Influencing direction: state backend choices, schema migrations, when a pipeline needs to be split or rebuilt.

Requirements

5+ years of professional software/data engineering experience, with strong Java expertise. Production experience with Apache Fli