Data Engineer

Description As a Data Engineer, you’ll develop and support secure, reliable data platforms on GCP. You’ll work with data analysts, data scientists, software engineers, and business partners to understand data needs and deliver solutions that are scalable, maintainable, and well documented. The role includes building batch and streaming pipelines, integrating data from multiple sources, and managing data across cloud storage, processing, and analytics services. You’ll also contribute to data quality, governance, monitoring, and cost optimization. Responsibilities • Design, develop, test, and maintain batch and streaming data pipelines on GCP. • Build data solutions using services such as BigQuery, Cloud Storage, Pub/Sub, Dataflow, Dataproc, and Cloud Composer, as appropriate. • Develop and optimize data transformations using SQL and Python. • Integrate data from databases, APIs, files, and other source systems. • Improve data quality, pipeline reliability, observability, and performance. • Apply data security, access control, privacy, and governance practices. • Monitor cloud resource usage and help optimize performance and cost. • Use version control, automated testing, and CI/CD practices to manage data engineering code. • Document data flows, technical designs, operational procedures, and data definitions. • Collaborate with cross-functional teams to translate business requirements into effective data solutions. Qualifications • Bachelor’s degree in computer science, data engineering, information systems, or a related field—or equivalent practical experience. • Experience developing data pipelines or data platforms in a cloud environment, preferably GCP. • Proficiency in SQL and Python. • Experience with BigQuery and cloud-based data storage and processing concepts. • Understanding of data modeling, ETL/ELT patterns, and data warehouse design. • Familiarity with batch and streaming data processing. • Experience with Git and software development practices such as testing, code review, and CI/CD. • Strong troubleshooting, communication, and problem-solving skills. • Ability to work collaboratively and explain technical concepts to varied audiences.