Databricks (Delta Lake, PySpark) AI Dev

Position Overview The Data & AI Engineer role is responsible for building scalable data pipelines, developing AI agents, and delivering end-to-end data and AI applications on the enterprise data platform, with a primary focus on Databricks.

This role bridges data engineering, AI engineering, and data science to deliver production-grade solutions that power analytics, automation, and intelligent applications across Suntory Beverage & Food’s global operations.

As part of the GHQ AAA team, you are expected to contribute to the organisation’s enterprise AI architecture by delivering reusable, scalable, and high-performance data and AI solutions.

Key Responsibilities

Data Pipeline Development Design, develop, and maintain scalable data pipelines using Databricks (Delta Lake, PySpark, workflows). Build robust ETL/ELT processes to ingest, transform, and serve data from multiple enterprise sources (SAP, external data, APIs).

Ensure data quality, reliability, and performance optimisation across pipelines. Implement data models aligned with analytics and AI use cases (e. g. , feature-ready datasets). Collaborate with data governance teams to ensure compliance with enterprise data standards.

AI Agent Development Design and develop AI agents and GenAI solutions (e. g. , knowledge assistants, automation agents) using Databricks and Azure AI capabilities. Implement Retrieval-Augmented Generation (RAG), prompt engineering, and orchestration logic for enterprise AI use cases.

Integrate agents with enterprise data sources, vector databases, and APIs. Collaborate with business teams to identify and deliver AI-driven automation and productivity use cases. Ensure scalability, performance, and responsible AI practices in agent deployment.

Databricks App Development Develop end-to-end data and AI applications using Databricks (e. g. , notebooks, dashboards, apps, APIs). Build interactive analytics or AI-driven applications for business users (e. g. , demand planning tools, decision support apps)