Big Data/AWS Data Engineer

Big Data / AWS Data Engineer About the Opportunity : We are looking for experienced Big Data / AWS Data Engineers with strong hands-on expertise in designing, developing, implementing, testing, and optimizing Big Data applications on AWS Cloud. The role involves working on both API and Batch applications using technologies such as Java, Scala, Kotlin, Python, Spark, Hadoop, Cassandra, Snowflake, and AWS services. Position Details : Client : Altimetrik Job Location : Hyderabad Work Model : Hybrid 2 days per week at client location Open Positions : 2 Client Interview : In-person interview for final discussion Key Responsibilities : - Analyze, design, code, implement, test, and support Big Data applications on AWS Cloud. - Develop and maintain both API-based and Batch applications. - Build scalable data processing solutions using Spark, Hadoop, HDFS, MapReduce, and related Big Data technologies. - Develop applications using Scala, PySpark, Java, Kotlin, and Python. - Develop and integrate REST APIs and SOAP Web Services. - Implement AWS-based Batch and API microservices using services such as EMR, EC2, ECS, S3, Airflow, Step Functions, and AWS API Gateway. - Perform performance tuning and optimization of batch applications across Spark, Hadoop, EMR, Java, and Python. - Develop automation pipelines for data ingestion and transformation in Snowflake using Spark. - Work with Linux environments and develop Shell/Python scripts for automation and operational activities. - Follow software engineering best practices for version control, build management, deployment, and CI/CD. - Collaborate with development, data, DevOps, and client teams to deliver high-quality solutions. Required Technical Skills : Big Data & Programming : - Strong hands-on experience with Big Data applications - Scala - PySpark - Java - Python - Kotlin - Spark - Hadoop - HDFS - MapReduce - Cassandra AWS Cloud : - Strong experience with AWS services, particularly : EMR, EC2, ECS, S3, Airflow, AWS Step Functions, and AWS API Gateway. - Experience developing both Batch applications and API-based microservices on AWS is required. Data Engineering : - Snowflake - Spark-based data processing - Data ingestion pipelines - Data transformation pipelines - Batch processing and optimization - Big Data application performance tuning APIs & Web Services : - REST APIs - SOAP Web Services - API-based microservices Linux & Scripting : - Linux - Shell scripting - Python scripting and automation Good-to-Have Skills : - Git, GitHub - Bitbucket - GitLab - IntelliJ IDEA - PyCharm - Maven - Gradle - Jenkins - Artifactory - AWS CodeCommit - AWS CloudFormation, and CI/CD pipeline development. Candidate Profile : The ideal candidate should have : - Strong hands-on development experience in Big Data and AWS Cloud. - Good programming skills in Scala, PySpark, and/or Java. - Practical experience building and optimizing Spark/Hadoop-based applications. - Experience working with AWS cloud-native services for Batch and API workloads. - Strong understanding of data ingestion, transformation, and processing pipelines. - Good problem-solving and debugging skills. - Ability to work effectively in a hybrid client environment. - Willingness to attend an in-person client interview for the final discussion. Work Location & Interview : Location : Hyderabad Work Model : Hybrid 2 days per week at client location Final Client Discussion : In-person