Data Engineer

We are looking for skilled Data Engineers to design, develop, implement, and maintain scalable data engineering solutions. The ideal candidate should have experience in data integration, ETL/ELT development, data pipelines, data warehousing, big data technologies, and cloud-based data platforms. Candidates with experience in one or more relevant data engineering technologies are encouraged to apply. Key Responsibilities • Design, develop, and maintain scalable data pipelines and data processing solutions. • Build and optimize ETL/ELT workflows for data ingestion, transformation, integration, and loading. • Extract, process, clean, validate, and transform structured, semi-structured, and unstructured data. • Develop and optimize SQL queries, stored procedures, data models, and database structures. • Work with data warehouses, data lakes, lakehouses, and enterprise data platforms. • Implement batch processing and real-time data streaming solutions where required. • Ensure data quality, integrity, consistency, reliability, and performance. • Work with cloud-based data engineering services and platforms. • Monitor, troubleshoot, and resolve data pipeline and production issues. • Implement data security, governance, access controls, and data management best practices. • Collaborate with data architects, analysts, data scientists, application developers, and business stakeholders. • Participate in code reviews, technical documentation, testing, deployment, and continuous improvement. Technical Skills Programming & Query Languages: Python, SQL, PySpark, Scala, Java, R, Shell Scripting. Big Data Technologies: Apache Spark, Hadoop, Hive, Kafka, Flink, HDFS. Data Warehousing & Lakehouse: Snowflake, Databricks, Delta Lake, Microsoft Fabric, Apache Iceberg, Amazon Redshift, BigQuery, Teradata. Data Modelling: Dimensional Modelling, Star Schema, Snowflake Schema, Data Modelling, Data Marts, Data Warehousing. Workflow Orchestration: Apache Airflow, Azure Data Factory, AWS Step Functions, Dagster, Prefect. Data Governance & Quality: Data Validation, Data Profiling, Data Lineage, Metadata Management, Master Data Management, Data Cataloguing, Data Security. Requirements • Bachelor's degree in Computer Science, Information Technology, Engineering, or a related discipline. • Relevant experience in Data Engineering, ETL Development, Data Integration, Data Warehousing, or Big Data. • Strong understanding of SQL, data structures, data modelling, and data processing concepts. • Hands-on experience with one or more programming languages and data engineering tools. • Experience developing, maintaining, or optimizing data pipelines and data integration workflows. • Familiarity with relational databases, data warehouses, or cloud data platforms. • Good analytical, troubleshooting, and problem-solving skills. • Ability to work independently and collaboratively in a team environment. • Good communication skills and the ability to work with technical and business stakeholders. Preferred Qualifications • Experience with AWS, Azure, or GCP data services. • Exposure to real-time data processing, streaming technologies, or distributed computing. • Knowledge of data governance, security, and compliance practices. • Experience with Agile/Scrum methodologies, CI/CD, and automated deployments. • Relevant cloud, data engineering, or platform certifications are an advantage. #J-18808-Ljbffr