Senior Software Engineer - Data Engineering

MoEngage is looking for a talented Senior Software Engineer - Data Engineering to join our team. If you have a proven track record in managing data infrastructure and pipelines on high-scale distributed systems, expertise in programming languages like Java or Python, and experience with cloud environments, you'll be instrumental in building and optimizing our data ecosystem. Key Requirements • Experience : A minimum in the data engineering field, demonstrating a track record of managing data infrastructure and pipelines on high scale distributed systems . • Programming Skills : Expertise in at least one high-level programming language, with a strong preference for candidates proficient in Java and Python . • Cloud Infrastructure : Proven experience in setting up, maintaining, and optimizing data infrastructures in cloud environments , particularly on AWS or Azure . • Tech Stack Proficiency : Hands-on experience with a variety of data technologies including, but not limited to: • Kafka for stream-processing • Kubernetes for container orchestration • AWS S3, Athena, and Glue for storage and ETL services • Spark for large-scale data processing • Debezium for change data capture • Apache Airflow for workflow management • Any one Streaming frameworks (Kstream/Flink/Samza/Spark streaming) • Data Processing : Demonstrable skills in cleansing and standardizing data from diverse sources such as Kafka streams and databases. • Query Optimization : Proficient in optimizing queries to achieve optimal performance with large datasets, minimizing processing times without sacrificing quality. • Problem-Solving Abilities : An analytical mindset with robust problem-solving skills , essential for identifying and addressing issues during data integration and schema evolution. • Cost Optimization Expertise : A keen eye for cost-saving opportunities without compromising on system efficiency. Capable of architecting solutions that are not only robust and scalable but also cost-effective, ensuring optimal resource utilization and avoiding unnecessary expenses in the cloud and data processing environments. Skills: Spark, Kafka, Kubernetes, Cloud Infrastructure, Apache Airflow, Java, Python Experience: 3.00-4.00 Years