Data Engineer

XPT Software Australia Pty Ltd | Contract Sydney, Australia | Posted on 10/30/2026 • XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company • XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains. • We have 120+technocrats in Australia working at our clientlocations. • XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe. • We have served100+ clients globally, fulfilling their onsite-offshoreneeds. Job Description Role Overview We are looking for a Senior Data Engineerwith strong hands-on experience in Azure Databricks and the Microsoft Azuredata ecosystem. The role will design, build, optimise, and support scalabledata pipelines and lakehouse solutions that enable analytics, reporting, datascience, and enterprise decision-making. What You'll Work On • Design,build, and optimise scalable batch and streaming data pipelines using AzureDatabricks, PySpark, Python, SQL, and Delta Lake. • Developreusable ingestion, transformation, validation, and publishing frameworks forlarge-scale enterprise datasets. • Implementmedallion or layered lakehouse patterns across raw, curated, andconsumption-ready data zones. • IntegrateAzure Databricks with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Key Vault, Azure DevOps, and related Azure services. • Developproduction-grade Databricks notebooks, jobs, workflows, libraries, and clusterconfigurations. • Createcomplex SQL and PySpark transformations, data-quality checks, reconciliationlogic, and error-handling routines. • OptimiseSpark workloads, storage layouts, partitioning, file sizes, caching, and queryperformance while controlling platform cost. • ImplementDelta Lake capabilities such as ACID transactions, schema enforcement, schemaevolution, time travel, and merge operations. • Buildautomated CI/CD pipelines for code promotion, configuration management,testing, and deployment across environments. • Applydata security and governance controls using role-based access, secretsmanagement, data masking, auditability, and controlled access patterns. • Monitordata pipelines, troubleshoot failures, perform root-cause analysis, and deliversustainable production fixes. • Partnerwith Architects, Business Analysts, Data Analysts, Product Owners, and businessstakeholders to convert requirements into reliable data products. • Providetechnical leadership, code reviews, engineering standards, and mentoring toother data engineers. KeySkills and Experience • 8+years of experience in data engineering, data integration, ETL/ELT, orlarge-scale data platform delivery. • Stronghands-on experience with Azure Databricks, including notebooks, jobs orworkflows, cluster configuration, debugging, and performance tuning. • Advancedproficiency in Python, PySpark, and SQL for production-grade data engineering. • Strongexperience with Delta Lake and lakehouse architecture, including reliableincremental processing and data-quality controls. • Practicalexperience with Azure Data Lake Storage Gen2 and Azure Data Factory. • Experienceimplementing CI/CD using Azure DevOps, Git, branching strategies, automatedtesting, and environment-based deployment controls. • Strongunderstanding of dimensional modelling, data warehousing, data lake design,metadata, lineage, partitioning, and file formats such as Parquet. • Experiencewith orchestration, monitoring, alerting, operational support, incidentanalysis, and service-level expectations. • Understandingof Azure identity, Key Vault, managed identities, networking, privateendpoints, and secure data-platform patterns. • Strongcommunication, documentation, problem-solving, code-review, and stakeholdercollaboration skills. Good to Have • Experiencewith Microsoft Purview, Unity Catalog, Azure Synapse Analytics, Event Hubs, orStructured Streaming. • Knowledgeof infrastructure as code using Terraform or Bicep. • Experiencewith dbt, Airflow, Kafka, Power BI, or machine-learning data pipelines. • Experiencein telecommunications or another high-volume enterprise environment. #J-18808-Ljbffr