Happiest Minds Technologies - Databricks Architect

: We are looking for an experienced Databricks Architect to design and lead enterprise-scale Lakehouse platforms on Databricks. The ideal candidate will have strong expertise in Databricks architecture, Apache Spark, Delta Lake, Unity Catalog, data governance, and cloud-native data platforms. This role involves defining platform standards, driving modernization initiatives, optimizing performance and cost, and enabling secure, scalable, and governed data solutions. Key Responsibilities : - Design and own the target-state Lakehouse architecture, including Unity Catalog, metastores, workspace strategy, data sharing, and lineage. - Define enterprise standards for Delta Live Tables (DLT), Databricks Workflows, streaming pipelines, CDC/SCD implementations, data contracts, and schema design. - Establish platform governance, security, and compliance standards, including RBAC/ABAC, row/column-level security, auditing, and data privacy controls. - Drive migration of legacy data warehouses and ETL platforms to Databricks using scalable migration frameworks and best practices. - Optimize platform performance through Spark tuning, Delta Lake optimization, Photon adoption, cluster policies, autoscaling, and workload management. - Implement cost governance and FinOps practices, including cluster optimization, resource utilization monitoring, and cost dashboards. - Define DataOps practices, CI/CD pipelines, automated testing, observability, SLIs/SLOs, and platform reliability standards. - Collaborate with data engineering, analytics, cloud, and business teams to deliver scalable and secure data solutions. - Mentor engineering teams and establish architectural guidelines, reusable frameworks, and best practices. Required Skills : - 10-16 years of experience in Data Engineering, Big Data, or Cloud Data Platforms, with significant experience in Databricks architecture. - Deep expertise in Apache Spark, Delta Lake, Databricks Lakehouse Platform, and performance optimization. - Strong experience with Unity Catalog, data governance, lineage, and enterprise security models. - Hands-on experience with Delta Live Tables (DLT), Databricks Workflows, streaming, CDC, and large-scale ETL pipelines. - Experience designing enterprise data architectures on AWS, Azure, or GCP. - Strong understanding of CI/CD, DataOps, monitoring, observability, and platform automation. - Experience with migration from traditional data warehouses or legacy ETL platforms to Databricks. - Excellent problem-solving, stakeholder management, and architecture documentation skills. Good to Have : - Experience with Delta Sharing, Lakehouse Federation, Databricks Marketplace, Model Serving, or Vector Search. - Knowledge of AI/ML workloads on Databricks. - Databricks or cloud platform certifications (AWS, Azure, or GCP).