Lumiq - Senior Data Engineer - Spark/Python

: We are seeking a Senior Data Engineer to design, build, and maintain enterprise-grade Data Warehouses and Lakehouses on AWS. You will be responsible for architecting scalable batch and real-time streaming data pipelines, optimizing big data processing workloads, and ensuring high data reliability across the organization. Key Responsibilities : - Design, develop, and optimize robust data pipelines using Spark, Python, and SQL. - Build and maintain real-time streaming architectures utilizing Apache Kafka. - Orchestrate complex workflows using Apache Airflow. - Architect and optimize cloud-native data warehouse and lakehouse solutions on AWS. - Collaborate with cross-functional teams to deliver high-performance, enterprise-ready data platforms. Required Experience & Skills : - Big Data & Processing : Advanced proficiency in Apache Spark, Python, and SQL. - Streaming : Hands-on experience with Apache Kafka and real-time streaming architectures. - Cloud Platform : Strong expertise in the AWS Ecosystem (S3, Redshift, Glue, EMR, IAM). - Orchestration : Proven experience building and managing DAGs in Apache Airflow. - Data Architecture : Track record of building scalable Enterprise Data Warehouses and Lakehouse architectures. Good to Have : - Experience with EMR on EKS and Kubernetes container orchestration. - Familiarity with AWS Lake Formation for governance and security management. - Relevant AWS Certifications (e.g., AWS Certified Data Engineer - Associate/Specialty, AWS Certified Solutions Architect). About LUMIQ : LUMIQ | Where Data Alchemy Meets FSI Expertise. LUMIQ is the #1 Data and Analytics company in the Financial Services and Insurance (FSI) space. Some of the largest FSIs - insurers, banks, AMCs, and NBFCs - trust our products and expertise to solve their data challenges. Currently, 40+ enterprises with $10 B in deposits/AUM, who collectively represent about 1B customers across the globe - work with LUMIQ to accelerate data-driven transformation. LUMIQ builds next-gen data technology products that help FSI enterprises organize, manage, and productize data effectively. With deep expertise across data engineering, data science, AI ML, and Cloud - LUMIQ's approach has often challenged the existing status quo - with many firsts. For ex : - Our teams built the first regulated data lake in India & Asia (2018).