Senior Data Engineer

Job Title: SeniorData Engineer Location: Remote Position Type: Full-Time Clearance Required: Secret Clearance About the company: At VivSoft, we aim to solve complex federal problems using emerging and open technologies in a collaborative and rewarding environment.

VivSoft is a diverse team of strategists, engineers, designers, and creators experienced in building high-performance, effective software, with a focus on impactful organisational design and software delivery dynamics.

We build secure Software Factories based on DoD reference designs and NIST Frameworks for Cloud and DevSecOps.

These factories deliver AI/ML Applications, Data Science Platforms, Blockchain and Microservices for DoD, Healthcare and Civilian Agencies Job Summary: We are seeking a Senior Data Engineer to support a United States Air Force (USAF) program responsible for building and operating a modern, scalable, and secure data platform.

This role will lead the design and optimization of an enterprise lakehouse architecture on AWS, leveraging Apache Iceberg, Apache Spark, and cloud-native technologies to enable advanced analytics, AI/ML initiatives, and operational reporting.

The ideal candidate will possess deep expertise in large-scale data platforms, distributed processing, data governance, and cloud infrastructure while providing technical leadership and mentoring across engineering teams.

Key Responsibilities

Architect, build, and manage a cloud-based lakehouse environment using Apache Iceberg, AWS S3, and AWS Glue Catalog. Develop and optimize Apache Spark data pipelines on EMR and Kubernetes environments. Design and implement event-driven data ingestion solutions using S3 events and Amazon SQS.

Optimize query performance across Athena, Trino, and Spark SQL environments. Orchestrate data workflows and ETL pipelines using Apache Airflow. Manage infrastructure deployment and automation using Terraform. Develop, publish, monitor, and maintain data products and analytical dashboards. Implemen