Senior Data Engineer

• Data Pipeline Infrastructure & Architecture 

• Design and implement scalable data architectures on cloud data platforms with high availability, security, and performance 

• Lead development of Data Lakehouse solutions 

• Collaborate with stakeholders to understand requirements and translate them into technical specifications 

• Pipeline Development & Optimisation 

• Build and maintain robust ETL/ELT pipelines using modern data engineering tools and frameworks 

• Optimise data processing workflows for performance, cost-effectiveness, and reliability 

• Implement automated data quality checks and monitoring systems to ensure data integrity 

• Data Systems Architecting & Solutioning 

• Design and architect comprehensive cloud-native Data & AI solutions aligned with business objectives and technical requirements 

• Lead cloud migration strategies and oversee implementation of complex multi-cloud environments 

• Drive innovation through integration of Data & AI capabilities into HDB’s Data & AI platform product architectures 

• Conduct technical assessments and recommend modernised approaches using cloud native technologies 

• Maintain architectural documentation 

• Cloud Platform Operations 

• Leverage Cloud Native Services to build and manage data infrastructure 

• Implement infrastructure as code practices using Terraform 

• Ensure compliance with security standards and data governance policies 

• Technical Leadership & Collaboration 

• Mentor junior data engineers and provide technical guidance on complex challenges 

• Participate in architectural reviews and contribute to data strategy evolution 

• Bachelor’s degree in computer science, Information Technology, Computer Engineering, or related field 

• Minimum 3 years of relevant experience in data systems architecture, data systems integration, and data pipeline setup at production scale 

• Good understanding of cloud computing principles including infrastructure as code, containerisation, microservices architecture, cloud security frameworks, identity and access management, network architecture, and distributed systems 

• Proven ability to translate business requirements into technical solutions 

• Excellent communication skills for presenting complex concepts to diverse audiences 

• Experience with cloud security frameworks, compliance requirements, and risk management 

• Experience in data domains (e.g. DataOps, Data Lakehouse) and AI/ML Domains (e.g. MLOps, LLMOps) 

• Strong Knowledge and Hands-on experience with SQL, Python and Apache Spark  

• Hands-on experience with Apache Kafka, Airflow, or similar technologies 

Good to Have: 

• Proficiency in Amazon Web Services (AWS) services 

• Relevant cloud certifications (e.g. AWS Solutions Architect Professional, AWS Data Engineer Associate) would be an advantage 

• Experience with Data & AI cloud-native services (e.g. Amazon SageMaker Unified Studio, Amazon Quick Suite, AWS S3, AWS Glue, AWS Lake Formation, AWS Bedrock, AWS Agent Core). 

• Familiarity with serverless computing, edge computing, and IoT architectures would be an advantage. 

• Experience with machine learning operations (MLOps) and ML model deployment pipelines 

• Knowledge of data governance frameworks and metadata management tools 

• Familiarity with data visualisation tools and business intelligence platforms