Numerator - Senior Data Engineer - Python/PySpark

About the Role : Key Responsibilities : - Design, build, and maintain scalable data ingestion and transformation pipelines using PySpark and Python. - Implement and manage data processing workloads on Databricks, ensuring optimal performance and cost efficiency. - Monitor data pipeline health, troubleshoot issues, and implement proactive reliability enhancements. - Document data models, pipeline designs, and operational procedures for knowledge sharing and compliance. Requirements : - 7 - 10 years of experience in data engineering roles - Experience in Pyspark and Python - Experience working with Microsoft Azure Cloud - Experience in Agile methodologies (Scrum/Kanban) - Experience with PySpark, Databricks - Experience working with Devops pipeline, preferably Azure DevOps Preferred Qualifications : - Bachelor's or master's degree in computer science, Information Technology, Data Science, or a related field. - Experience working in a support focused role - Knowledge/experience in AI ML techniques - Certification in relevant Data Engineering discipline or related fields.