Senior Data Engineer

Senior Data Engineer - Java, AWS, Kafka, Kubernetes

SC Cleared | London (Hybrid) | Inside IR35 | Contract until December 2027

Are you an experienced Senior Data Engineer with a strong Java software engineering background looking to work on a high-profile UK Government AI & Data programme?

We're recruiting for a Senior Data Engineer to support a major public sector organisation delivering critical data solutions within a secure environment.

This is an opportunity to work with modern cloud-native technologies, build scalable data processing solutions and contribute to systems that support essential public services.

The Role

You'll work within an established engineering environment, developing and maintaining robust data solutions using Java and modern cloud technologies.

The role combines hands-on software engineering with data engineering, making it particularly suitable for someone who has developed production-grade Java applications alongside data pipelines and distributed processing systems.

Key Responsibilities

• Design, develop and maintain scalable data pipelines and processing solutions.

• Build reliable, high-performance applications using Java.

• Develop and enhance data ingestion, transformation and integration processes.

• Work with cloud-based infrastructure and services, particularly AWS.

• Support event-driven and distributed processing architectures.

• Apply data modelling, data quality and data life cycle management principles.

• Collaborate with software engineers, data engineers, architects and other delivery teams.

• Contribute to technical design, code reviews, testing and continuous improvement.

• Deliver solutions using Agile engineering practices.

Essential Skills and Experience

• Advanced Java development experience, including hands-on delivery of production systems.

• Strong commercial experience in Data Engineering.

• Experience designing and developing data pipelines, ETL processes and data integration solutions.

• Strong software engineering fundamentals and experience building scalable, maintainable applications.

• Knowledge of data modelling, data quality and data life cycle management.

• Experience working within Agile development teams.

• Active UK SC security clearance.

Highly Desirable

• AWS - experience building or supporting cloud-native data solutions.

• Apache Kafka - experience with event streaming, messaging or Real Time data processing.

• Kubernetes - experience with container orchestration and distributed application deployment.

• Experience developing Java-based data processing or event-driven applications.

Additional Advantageous Experience

• Apache Spark, Flink, Hadoop or other Big Data technologies.

• Working with high-volume or complex enterprise datasets.

• Experience delivering solutions within government, defence or other secure environments.

Why Consider This Opportunity?

• Long-term contract running until December 2027.

• Work on a significant UK public sector AI & Data programme.

• Modern technology stack involving Java, AWS, Kafka and Kubernetes.

• Opportunity to contribute to secure, business-critical data platforms.

• Hybrid London working arrangement.

Interested?

If you're an SC-cleared Senior Data Engineer with strong Java experience and exposure to modern cloud and data technologies, we'd be keen to hear from you.

Apply now or get in touch for a confidential discussion.

Please note: Active SC clearance is required for this position. The role requires three days onsite in London each week and falls inside IR35.