Azure Data Engineer

Role Overview We are looking for experienced Azure Databricks Data Engineers with strong hands-on expertise in Apache Spark/PySpark, Python, SQL, and Azure Data Factory (ADF) . The candidate will be responsible for designing, developing, and optimizing scalable data engineering solutions on the Azure cloud platform. Mandatory Skills • Strong hands-on experience with Azure Databricks • Strong knowledge of Apache Spark and/or PySpark • Proficiency in Python • Strong SQL development and query optimization skills • Hands-on experience with Azure Data Factory (ADF) • Experience developing and maintaining scalable ETL/ELT data pipelines • Good understanding of Azure data engineering concepts and cloud-based data platforms Key Responsibilities • Design, develop, and maintain data pipelines using Azure Databricks and ADF • Develop efficient data processing solutions using PySpark/Spark and Python • Write complex SQL queries for data transformation, validation, and analysis • Build scalable ETL/ELT workflows for large datasets • Optimize Spark jobs, Databricks notebooks, and data pipelines for performance • Integrate data from multiple sources into Azure-based data platforms • Implement data quality, validation, error handling, and monitoring mechanisms • Troubleshoot production issues and provide timely resolutions • Collaborate with data architects, analysts, developers, and business stakeholders • Follow best practices for code quality, security, performance, and maintainability Experience Requirements For 5+ Years • 5+ years of overall experience in data engineering • Strong hands-on experience in Azure Databricks, PySpark/Spark, Python, SQL, and ADF • Experience working on enterprise-scale data pipelines For 9+ Years • 9+ years of overall experience in data engineering • Strong expertise in Azure Databricks and modern Azure data engineering • Proven experience designing and optimizing large-scale data pipelines • Ability to lead technical discussions and mentor junior engineers Preferred Skills • Azure Data Lake Storage (ADLS) • Delta Lake • Databricks Workflows • Azure DevOps / CI-CD • Git • Data warehousing concepts • Experience with Agile/Scrum methodologies Skills:- Azure Databricks, Azure data factory, Python, PySpark, Apache Spark and SQL