Lead/Senior Data Engineer - ETL/Python

Working Details : Working Days : Monday - Friday Job Timing : 9AM - 6PM Position Details : Position / Designation : Lead / Senior Data Engineer - Hybrid Qualification : Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical field Years of Experience : 6 to 9 Years Employment Type : Permanent Full Time Location: Rajiv Gandhi Salai, Taramani, Chennai Number of posts: 1 Gender: Male / Female Joining Date: Immediate to 1 Month Selection Process : 1. Total 3 Technical rounds. 2. 2 rounds evaluation with LV and 1 with client. Job Role & Responsibility : 1. Architecture & Design : - Design and architect scalable, high-throughput, and reliable end-to-end data systems (databases, warehouses, and big data platforms) adhering to data governance, privacy, and compliance standards. 2. Lead & Technical Delivery : - Drive the end-to-end technical delivery of complex data initiatives. - Translate broad business requirements into robust, production-grade technical designs. 3. Pipeline Engineering : - Lead the implementation and optimization of complex ETL/ELT data pipelines using SQL, PySpark, Hive, and Airflow across large-scale distributed environments. 4. Team Leadership & Mentorship : - Provide technical leadership, code reviews, and architectural guidance to junior and mid-level data engineers to enforce best practices and quality standards. 5. Stakeholder Management : - Collaborate with business leadership, product managers, and enterprise architects to align data strategies with overall organization goals. 6. AI & Innovation : - Leverage AI / GenAI tools to streamline pipeline generation, query optimization, automated test suite generation, and metadata management. Skills : Primary Skills : - Advanced PySpark, Advanced SQL, Data Architecture & Design, Big Data Ecosystem (Hadoop, HDFS, Hive), Airflow / Workflow Orchestration, Data Warehousing (Snowflake / Redshift / BigQuery). Secondary Skills : - Enterprise Cloud Platforms (AWS / GCP / Azure), CI/CD & DevOps for Data Platforms, GenAI / AI-driven pipeline automation, Data Governance & Quality Frameworks, Streaming (Kafka / Spark Streaming). Key Skills: Pyspark, Hadoop, HDFS, Hive, Airflow / Workflow Orchestration, Bigquery, Redshift, SQL, Snowflake, Cloud, Spark, ETL, Advanced SQL, Python.