Senior Data Engineer

: The Senior Data Engineer will be responsible for designing, developing, and maintaining scalable cloud-based data platforms and data pipelines. The role requires strong expertise in AWS, Snowflake, Databricks, Python, SQL, and PySpark, along with experience in batch and real-time data processing. The candidate will collaborate with cross-functional teams to build reliable, secure, and highperformance data solutions that support analytics, reporting, and AI/ML initiatives. Key Responsibilities: • Design, develop, and maintain scalable data pipelines and ETL/ELT frameworks. • Build and optimize batch and real-time streaming solutions using PySpark and cloud-native services. • Develop and manage enterprise data platforms using AWS, Snowflake, and Databricks. • Design and implement robust data models, data lakes, data warehouses, and lakehouse architectures. • Integrate data from multiple sources while ensuring data quality, governance, and security. • Optimize performance and scalability of data processing workloads. • Collaborate with Data Scientists, Analysts, Architects, and business stakeholders to deliver data solutions. • Implement monitoring, alerting, and automated data validation processes • Contribute to architecture reviews, technical documentation, and best practices. • Mentor junior engineers and participate in code reviews. • Stay updated on emerging cloud and data engineering technologies. Required Skills and Experience: • 5-8 years of relevant experience in Data Engineering, Big Data, and Cloud Data Platform development. • Strong proficiency in Python, SQL, and PySpark. • Hands-on experience designing, developing, and maintaining scalable data platforms on AWS Cloud. • Strong expertise in Snowflake and Databricks for enterprise-scale data engineering solutions. • Experience building and optimizing batch and real-time streaming data pipelines. • Strong understanding of Data Modeling concepts, including Dimensional • Modeling, Star Schema, Snowflake Schema, and Data Vault. • Experience designing and implementing ETL/ELT pipelines for large-scale enterprise data platforms. • Strong expertise in Apache Spark and distributed data processing frameworks. • Experience building and managing Data Lakes, Data Warehouses, and Lakehouse architectures. • Hands-on experience with AWS services such as S3, Glue, EMR, Lambda, Redshift, Athena, EC2, ECS/EKS, Kinesis, MSK, Step Functions, CloudWatch, IAM, RDS, DynamoDB, EventBridge, SNS, and SQS. • Experience integrating data from multiple structured and unstructured data sources. • Strong understanding of data governance, security, data quality, and performance optimization best practices. • Experience with version control systems such as Git and collaborative development workflows. • Proficiency with CI/CD pipelines and software engineering best practices. • Strong problem-solving skills with the ability to debug, troubleshoot, and resolve complex data engineering challenges. • Ability to design scalable and maintainable data solutions using modern architectural patterns and best practices. • Proven ability to work collaboratively in cross-functional teams, including participation in code reviews, technical discussions, and knowledge-sharing initiatives. • Experience working in Agile/Scrum delivery environments. • Strong communication skills and ability to interact with business and technical stakeholders. • Experience in Healthcare, Pharma, or Real-World Evidence (RWE) data environments (good to have). • Experience with Infrastructure as Code (Terraform/CloudFormation) (good to have). • Experience with Docker and Kubernetes (good to have). • Knowledge of Data Mesh, CDC, and Event-Driven Architectures (good to have). • Exposure to MLOps and AI/ML platform integration (good to have). • AWS, Snowflake, or Databricks certifications (good to have).