Lead Data Engineer

Job Title: Lead Data Engineer Location: Toronto , ON (Onsite) Job Type: Contract Responsibilities: • Lead the design, development, and implementation of scalable data engineering solutions on Microsoft Azure. • Architect and develop ETL/ELT pipelines using Azure Data Factory (ADF) and Azure Databricks. • Build and optimize large-scale data processing frameworks using PySpark and Spark SQL. • Design and maintain enterprise data lakes and Delta Lake architectures. • Develop robust data models to support analytics, reporting, and business intelligence initiatives. • Optimize data pipelines for performance, scalability, reliability, and cost efficiency. • Implement data quality, governance, security, and compliance best practices. • Design and support CI/CD pipelines for automated deployment of data engineering solutions. • Collaborate with solution architects, business analysts, data scientists, and stakeholders to translate business requirements into technical solutions. • Troubleshoot complex production issues and ensure high availability of enterprise data platforms. • Mentor junior and mid-level data engineers while promoting engineering best practices. • Participate in code reviews, architecture discussions, and technical design sessions. Skills : • Lead the design and delivery of scalable data platforms using Azure Data Factory, Databricks, PySpark, SQL, and Delta Lake. Architect ETL/ELT pipelines, data lakes, and analytics solutions • Drive performance optimization, governance, CI/CD, and stakeholder engagement while mentoring teams and ensuring reliable enterprise data products • Strong experience with Azure Databricks and Azure Data Factory (ADF). • Expertise in PySpark, Apache Spark, and Spark SQL. • Strong SQL programming and database optimization skills. • Hands-on experience with Delta Lake, data lakes, and modern data architectures. • Experience designing enterprise-scale ETL/ELT pipelines. • Strong understanding of Azure cloud services and data engineering best practices. • Experience implementing CI/CD pipelines using Azure DevOps or Git-based deployment tools. • Knowledge of data governance, data security, and performance tuning. • Experience working in Agile/Scrum environments.