Job Description
Overview About Business Unit: The Automotive Practice at Epsilon is a rapidly growing team, driving growth for major players in the automotive industry - from Original Equipment Manufacturers (OEMs) to dealerships across North America. Part of a 1,600-member multinational team, the practice provides the automotive world's largest service reminder platform, alongside agency services and digital media solutions. A leader in the automotive space, the team supports over 50% of auto dealerships in North America and maintains relationships with over 280 million customers. Home to innovation and ground breaking technology, our Auto team leads the game in developing outstanding software and solutions for hyper-personalized digital marketing. We are looking for a highly skilled Databricks Data Engineer to design, build, and maintain our end-to-end Databricks data platform. The candidate will be responsible to Develop and builds, maintains, and optimizes data pipelines (ETL/ELT) using PySpark, SQL, and Delta Lake within cloud environments (Azure, AWS, or GCP) . They work under mentorship to automate data workflows, perform troubleshooting, and ensure data quality, often contributing to Medallion Architecture (Bronze/Silver/Gold) layers. Click here to view how Epsilon transforms marketing with 1 View, 1 Vision and 1 Voice. Responsibilities Develop and maintain data ingestion and transformation pipelines using Databricks Notebooks and PySpark. Implement ETL/ELT workflows using Delta Lake, Autoloader, and Delta Live Tables (DLT). Write complex SQL queries for data manipulation and optimize Spark jobs for performance. Work with senior engineers to troubleshoot data issues and apply data engineering best practices. Document data mappings, workflows, and unit tests for pipelines. Use Git for code management and collaboration. Explore ML and AI workloads using Cursor or any AI tool ML (nice to have). Qualifications Proficiency in Python (PySpark) and SQL is essential. Familiarity wi
