Custom Software Engineer - Microsoft Azure Data Services

As a Custom Software Engineer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with various stakeholders to gather requirements, developing innovative solutions, and ensuring that applications are optimized for performance and usability. You will also participate in code reviews and contribute to the continuous improvement of development practices, ensuring that the applications you build align with the strategic goals of the organization. Roles & Responsibilities: • Design, develop and maintain / support advanced ETL pipelines on Azure cloud technologies to ingest & integrate data coming from multiple sources and make it ready for analytics, data science business needs. • Analyse ML algorithms that could be used to solve problems and rank them by their success probability. • Ensure the end users of the Data, whether they be a Analysts or Data Scientists or Executive, can get accurate data which can help them make insightful decisions. • Produce optimised dimensional database • Designs to support hundreds of users querying millions of transactions at performance ranging from minutes down to sub-second. • Build the next generation platform using the latest Cloud-based technology. • Contribute to the end-2-end software development life cycle. Professional & Technical Skills: • Ability to work across structured, semi-structured and unstructured data, extracting information and identifying linkages across disparate data sets. • Hands on experience with at least one of the programming languages: Scala, Python • Experience of working with traditional data warehousing, Azure Data Factory, Azure Data Bricks, Azure Data Lake Storage, Azure Devops, Azure storage services, Azure data services and equivalent technologies • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases. Experience in at least two database technology such as: • Traditional RDBMS (Postgres, MS SQL Server, Oracle) • Experience in Distributed Processing using Databricks (preferred) or Apache Spark • Good knowledge with Scala/Python and basic libraries for machine learning such as scikit-learn and pandas. • Proficient at working with large and complex code bases using Github, Gitflow, Fork/Pull Model and CI/CD. • Exposure to webservice (SOAP/REST) development skills. • Working experience in Agile methodologies (SCRUM, XP, Kanban) Good to have - Exposer to IOT central/IOT hub and Exposer to work with AKS.