Technical Lead - Data Engineer

Data Engineering We are seeking a highly skilled and experienced Data Engineering with strong expertise in Microsoft Azure data services, enterprise data integration, and modern data lake architectures .

In this role, you will lead the design, development, and optimization of scalable data pipelines responsible for ingesting and transforming data from enterprise systems including Microsoft platforms, SAP, CRM systems, retail systems, and external third-party data sources into a centralized data platform to support downstream applications, analytics, AI, and agentic use cases.

You will play a critical role in building a robust, high-performance data platform using Azure-native services , ensuring data quality, reliability, and near real-time availability for business operations.

The ideal candidate will have hands-on experience building Medallion Architecture (Bronze, Silver, Gold) data lakes , integrating diverse enterprise data sources, and enabling AI-powered solutions using the Microsoft ecosystem.

Responsibilities

Design and implement scalable ETL/ELT pipelines to ingest data from enterprise systems including Microsoft applications, SAP, CRM platforms, retail systems, and external third-party sources into Azure-based data platforms.

Design, build, and maintain enterprise-scale Medallion Architecture Data Lakes (Bronze, Silver, Gold) using Azure Data Lake Storage, Microsoft Fabric, Databricks, or similar technologies.

Develop and optimize data processing workflows using Azure Databricks, Apache Spark / PySpark , Azure Data Factory, Microsoft Fabric, and other Azure-native services.

Create denormalized, API-ready, analytics-ready, and AI-ready data models aligned with downstream applications, microservices, reporting, and agentic AI consumption patterns.

Implement idempotent processing, CDC merge (upsert) strategies, incremental processing, and data reconciliation mechanisms to ensure consistency across batch and streaming pipelines. Leverage Azure Data Lake Stora