ValueMomentum Software - Senior Data Engineer - Azure Data Factory

Key Responsibilities: - Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF). - Build and optimize data ingestion, transformation, and processing pipelines for structured and unstructured data. - Develop scalable data solutions using Azure Databricks, Azure Data Lake Storage (ADLS), and Azure Synapse Analytics. - Write efficient and optimized SQL, Python, and PySpark code for data transformation and processing. - Design and implement analytical data pipelines for big data and relational database environments. - Collaborate with cross-functional teams to understand business requirements and translate them into scalable data solutions. - Ensure data quality, performance, scalability, and reliability across the data platform. - Monitor, troubleshoot, and optimize data workflows and processing jobs. - Implement best practices for data engineering, security, governance, and performance optimization. - Support deployment automation using DevOps and CI/CD pipelines. Required Skills & Qualifications: Must-Have Skills: - Strong experience in Azure Data Factory (ADF) for ETL pipeline development. - Hands-on expertise with Azure Databricks and PySpark. - Experience working with Azure Data Lake Storage (ADLS). - Strong knowledge of Azure Synapse Analytics and Azure SQL Database. - Proficiency in SQL, Python, and PySpark. - Experience building data ingestion, transformation, and analytical pipelines. - Strong understanding of big data processing and relational database concepts. - Experience designing scalable, high-performance data engineering solutions. - Strong analytical, debugging, and problem-solving skills.