Data Engineer - Python/SQL

Functional Reporting : Functional reporting : Group leader Administrative reporting : DPM LOCATION : Gurugram, Haryana Educational Qualification : - B.E / B. Tech in Computer Science/IT/ or Masters in Computer Application(MCA) - Minimum 60 % marks - Minimum 3 Years work experience in Big Data/Cloud Computing/ Data Engineering/ Analytics related projects Desirable : - Certification in Big Data/Data Science/Cloud Computing or equivalent Experience : - Minimum : 5 Year(s), Maximum : 8 Year(s) Job Role & Responsibilities : - Data Engineering & Quality Management : Design, implementation, and management of data platform, data pipelines to help fulfill data access and aggregation functionality for users in cloud environment. - Develop and Maintain Data Pipelines : Design, build, and manage scalable data pipelines for data extraction, transformation, and loading (ETL). - Data Integration : Integrate data from various sources, ensuring data quality and consistency. - Optimize Data Systems : Identify and implement process improvements to enhance data reliability, efficiency, and quality. - Collaborate with Teams : Work with data scientists, analysts, and other stakeholders to support their data infrastructure needs. - Monitor and Troubleshoot : Monitor data pipelines and systems, troubleshoot issues, and ensure data integrity. - Documentation : Maintain comprehensive documentation of data processes and systems. Technical/ Functional Competencies : - Experience : 3 - 5 years of experience in data engineering or a related field. - Technical Skills : Proficiency in Py-Spark (Must Have), SQL, Python, and data platform like Databricks. - Storage Management : Experience in storage mechanism in AWS S3, Layers of Medallion Architecture, OTF like Delta, iceberg etc., Hand-Ons in Data schema like Star, snowflake etc. - Good knowledge next generation ETL/ELT tools and data consolidation frameworks that enable capabilities within the data ecosystem. - Ability to work independently or within a team, proactively, in a fast-paced AGILE-SCRUM environment. Desired but not essential : - Experience in IoT time series data domain. - Exposure to automotive data. Behavioral competencies : - Effective team player. - Study industry trends (global and domestic) and apply it in work-area. - Co-ordinate department activities and represent department / MSIL in forums. - Good communication and presentation skills.