ValueLabs - Senior PySpark Data Engineer

Key Responsibilities : - Define and drive enterprise data architecture, data strategy, and technology roadmaps aligned with business objectives. - Design and implement scalable, end-to-end data management and analytics solutions using modern Big Data platforms. - Lead the transformation of traditional data warehouse environments into modern Big Data-based architectures. - Design and implement dimensional data models, including Star and Snowflake schemas, slowly changing dimensions (SCDs), role-playing dimensions, and dimensional hierarchies. - Define and implement cloud-native data architectures, deployment strategies, and workload migration approaches. - Establish and enhance data quality, data profiling, data governance, data security, metadata management, data classification, and data archival practices. - Identify and mitigate data-related risks while ensuring compliance with organizational security policies and applicable banking regulations. - Develop, maintain, and enhance CI/CD pipelines and automated deployment processes for data engineering solutions. - Collaborate with business stakeholders, enterprise architects, infrastructure teams, and internal IT partners to deliver strategic data initiatives. - Drive project planning, estimation, scheduling, and execution across multiple initiatives in a complex, matrixed environment. - Provide technical leadership, coaching, and mentoring to data engineering teams while promoting engineering best practices. - Manage multiple priorities and deliver high-quality solutions within tight deadlines and changing business requirements. Experience - 8+ yrs Experience : More than 8+ years of experience in information technology, with 3+ years spent in data engineering, architecture and technology solutions definitions and implementations. Extensive experience in banking services or related domain Knowledge & Skill : Expertise in Data Architecture, Data Strategy and Roadmap for large and complex organization and systems and implemented large scale end-to-end Data Management & Analytics solutions. Experience in transforming traditional Data Warehousing approaches to Big Data based approaches and proven track record of managing risks and data security. Expertise with DW Dimensional modeling techniques, Star & Snowflake schemas, modeling slowly changing dimensions and role playing dimensions, dimensional hierarchies, and data classification. Experiences in cloud native principals, designs and deployments. Extensive experience working with and enhancing Continuous Integration (CI) and Continuous Development (CD) environments. Expertise in Data Quality, Data Profiling, Data Governance, Data Security, Metadata Management, and Data Archival Define workload migration strategies using appropriate tools. Drive delivery in a matrixed environment working with various internal IT partners. Demonstrated ability to work in a fast paced and changing environment with short deadlines, interruptions, and multiple tasks/projects occurring simultaneously. Must be able to work independently and have skills in planning, strategy, estimation, scheduling. Strong problem solving, influencing, communication, and presentation skills, self-starter. Experience with data processing frameworks and platforms (Hadoop, Presto, Tez, Hive, Spark etc. Hands-on experience in work with bigdata platforms and related programming languages (e. Linux, Python, PySpark, GIT, Jenkins). Behavioral Competencies : Business Acumens Organization leadership skills Coaching and mentoring Excellent analytical skills Demonstrate critical and systems of thinking ability. Ability to negotiate and influence Disciplined, organized Flexibility / Adaptability Visionary Performance driven -