Data Engineer
- MI5
- 20/05/2026
Full time
Information Technology
Telecommunications
Java
SQL
Oracle
Python
Data Scientist
Business Analyst
Business Intelligence
Part Time IT
Job Description
Salary: £66,332. This includes a £3,000 non-consolidated market rate supplement. There is also an opportunity to qualify for additional allowances and a skills-related payment under the Government Digital and Data Profession Capability Framework (formerly the Digital Data and Technology (DDaT) Profession Capability Framework.)
Flexible Working: We offer a strong work life balance, supporting full time, part time, and flexible hours. You'll be predominantly office based, with working from home limited and subject to business need.
About us MI5 keeps the country safe from serious threats like terrorism and attempts by states to harm the UK, its people and way of life. We carry out investigations by obtaining, analysing, and assessing intelligence, and then work with a range of partners including MI6 and GCHQ to disrupt these threats. Through our protective security arm, we provide advice and guidance to government, businesses, and other organisations about how to keep themselves safe. A role in MI5 means you'll do unique and challenging work in a supportive and encouraging environment, making a real difference to UK national security.
The role Technology sits at the heart of everything we do at MI5. We're looking for people with hands on experience with data engineering: at its core designing and managing data pipelines and storage, be it in the extraction, transformation, integration or loading of multiple sources of data, to take on core roles within a multi disciplinary product team. As a data engineer, you will work alongside other engineering disciplines, product owners, business analysts, delivery managers and data scientists, to build and run secure data applications and data products.
You'll use and promote Agile technical practices and methodologies to deliver digital products that are core to MI5's operation, making use of modern standards and approaches whilst using cloud technologies, as well as on premises platforms and long established technologies and frameworks.
You'll gain experience in a fast paced environment as part of a forward thinking team and strive to develop elegant digital solutions. Managing tasks, you'll be responsible for the design and implementation of moderate to complex data flows, to connect operational systems, data for analytics, or business intelligence (BI) systems.
Ensuring our customer's needs are represented at every stage, you'll support the products owned by the team, working with users to deliver new features, identify and fix defects and develop automated tests to maintain the assurance of our products.
You'll frame problems and set priorities for yourself and your team, taking ownership of large problems, breaking them down and working with the team to deliver new features throughout the engineering lifecycle. As a Data Engineer, you'll focus on creating professional and comprehensive digital experiences and share a collective responsibility in keeping the country safe, by staying one step ahead in an evolving digital world.
You'll help set the standard for what good looks like, sharing digital and innovative ways of working with the team and across the skills community. Importantly, you'll have a passion for sharing knowledge, whether it's a customer or colleague - you'll help them to understand what great engineering looks like. You'll also participate in our Communities of Practice, and encourage cross organisation initiatives to help build our community of engineers, whilst mentoring and developing junior members of the team.
Our tech stacks vary between products using a range of databases including SQL and No SQL and commercial and custom built technologies. We embrace a DevSecOps (Development, Security and Operations) mindset, allowing our teams to automate, run and support what we build in a secure manner.
Whether it's building a platform to enable the analysis of data systems to track the progress of our investigations, ingesting new complex partner data, developing robust and scalable capabilities for our Data Science teams, delivering compliance services to ensure we are operating legally, or tooling to improve our developer experience, the nature of our work presents unique and fascinating engineering challenges unique to MI5.
About you You'll be able to communicate information clearly and effectively, influencing technical and non technical colleagues. You'll be able to define and shape problems while working with a wide range of stakeholders to meet customer needs. You'll look to build leadership qualities, supporting others to upskill and build their career in an inclusive environment, while continuously looking for opportunities to develop and learn new engineering practices and approaches.
You'll be able to demonstrate a proficiency in developing data solutions and data integration designs. Crucially, you'll be comfortable learning and using a broad range of technologies whilst utilising modern approaches, such as the use of continuous integration and delivery (CI/CD), and use of cloud services. You'll have an awareness of good design patterns and how to implement them appropriately with security in mind, and demonstrate proficiency in working with an agile mindset.
You'll have experience delivering moderate to complex data flows as part of a development team in collaboration with others. You'll be confident using data processing and integration technologies for example any of: AWS Glue, Fivetran, Airbyte, Apache NiFi, SAS DI Studio, Apache Spark or other equivalents.
You can implement, deliver, and translate across common data models, for example across unstructured data, document formats, tabular and relational models, or formats to support geo enabled applications or graph database structures as well as knowledge using their equivalent differing data file formats for example JSON, XML, CSV, Parquet, Avro, etc. You can also understand how to manage metadata across your development teams systems and beyond, with a practical knowledge of tracking data provenance and data lineage.
You'll work to agreed and recognised standards to build solutions using various traditional or big data languages such as: Python, Java, SQL, Scala, PL/SQL (Oracle), SAS Macro Language, etc. whilst using various data tools for example Jupyter, Hue (Hive/Impala), Kibana (Elastic Search), SageMaker (AWS), SAS for example.
Training and development We're committed to developing our people and recognise this will look different for everyone. We use the Government Digital and Data Profession Capability Framework as a compass to provide guidance and tools to support you, and which recognises and rewards specialist expertise.
There'll be opportunities to gain qualifications and pursue specialist pathways, as well as undertaking tailored training, coaching, and mentoring. Individuals can also attend hackathons and have up to 20% innovation and personal development time.
As you develop, grow, and maintain your technical skills, you can progress through the Government Digital and Data pay points. You'll need to regularly demonstrate and accredit your skills based on the framework. If you don't maintain the required skills, you'll be given the opportunity and support to rectify this, as you could revert to a lower level of capability and pay.
Rewards and benefits You'll receive a starting salary of £66,332 dependent on skills, experience, and location plus other benefits including:
We're Disability Confident MI5 is proud to have achieved Leader status within the Department for Work and Pensions' Disability Confident scheme. This is aimed at encouraging employers to think differently about disability and take action to improve how they recruit, retain and develop disabled people. As a Disability Confident Leader, we aim to ensure that a fair and proportionate number of disabled applicants who best meet the essential minimum criteria for this position, will be offered an interview, if it is practical for us to do so. (This is known as the Offer of an Interview.) To secure an interview for this role, the minimum criteria (in order of application process) are:
Flexible Working: We offer a strong work life balance, supporting full time, part time, and flexible hours. You'll be predominantly office based, with working from home limited and subject to business need.
About us MI5 keeps the country safe from serious threats like terrorism and attempts by states to harm the UK, its people and way of life. We carry out investigations by obtaining, analysing, and assessing intelligence, and then work with a range of partners including MI6 and GCHQ to disrupt these threats. Through our protective security arm, we provide advice and guidance to government, businesses, and other organisations about how to keep themselves safe. A role in MI5 means you'll do unique and challenging work in a supportive and encouraging environment, making a real difference to UK national security.
The role Technology sits at the heart of everything we do at MI5. We're looking for people with hands on experience with data engineering: at its core designing and managing data pipelines and storage, be it in the extraction, transformation, integration or loading of multiple sources of data, to take on core roles within a multi disciplinary product team. As a data engineer, you will work alongside other engineering disciplines, product owners, business analysts, delivery managers and data scientists, to build and run secure data applications and data products.
You'll use and promote Agile technical practices and methodologies to deliver digital products that are core to MI5's operation, making use of modern standards and approaches whilst using cloud technologies, as well as on premises platforms and long established technologies and frameworks.
You'll gain experience in a fast paced environment as part of a forward thinking team and strive to develop elegant digital solutions. Managing tasks, you'll be responsible for the design and implementation of moderate to complex data flows, to connect operational systems, data for analytics, or business intelligence (BI) systems.
Ensuring our customer's needs are represented at every stage, you'll support the products owned by the team, working with users to deliver new features, identify and fix defects and develop automated tests to maintain the assurance of our products.
You'll frame problems and set priorities for yourself and your team, taking ownership of large problems, breaking them down and working with the team to deliver new features throughout the engineering lifecycle. As a Data Engineer, you'll focus on creating professional and comprehensive digital experiences and share a collective responsibility in keeping the country safe, by staying one step ahead in an evolving digital world.
You'll help set the standard for what good looks like, sharing digital and innovative ways of working with the team and across the skills community. Importantly, you'll have a passion for sharing knowledge, whether it's a customer or colleague - you'll help them to understand what great engineering looks like. You'll also participate in our Communities of Practice, and encourage cross organisation initiatives to help build our community of engineers, whilst mentoring and developing junior members of the team.
Our tech stacks vary between products using a range of databases including SQL and No SQL and commercial and custom built technologies. We embrace a DevSecOps (Development, Security and Operations) mindset, allowing our teams to automate, run and support what we build in a secure manner.
Whether it's building a platform to enable the analysis of data systems to track the progress of our investigations, ingesting new complex partner data, developing robust and scalable capabilities for our Data Science teams, delivering compliance services to ensure we are operating legally, or tooling to improve our developer experience, the nature of our work presents unique and fascinating engineering challenges unique to MI5.
About you You'll be able to communicate information clearly and effectively, influencing technical and non technical colleagues. You'll be able to define and shape problems while working with a wide range of stakeholders to meet customer needs. You'll look to build leadership qualities, supporting others to upskill and build their career in an inclusive environment, while continuously looking for opportunities to develop and learn new engineering practices and approaches.
You'll be able to demonstrate a proficiency in developing data solutions and data integration designs. Crucially, you'll be comfortable learning and using a broad range of technologies whilst utilising modern approaches, such as the use of continuous integration and delivery (CI/CD), and use of cloud services. You'll have an awareness of good design patterns and how to implement them appropriately with security in mind, and demonstrate proficiency in working with an agile mindset.
You'll have experience delivering moderate to complex data flows as part of a development team in collaboration with others. You'll be confident using data processing and integration technologies for example any of: AWS Glue, Fivetran, Airbyte, Apache NiFi, SAS DI Studio, Apache Spark or other equivalents.
You can implement, deliver, and translate across common data models, for example across unstructured data, document formats, tabular and relational models, or formats to support geo enabled applications or graph database structures as well as knowledge using their equivalent differing data file formats for example JSON, XML, CSV, Parquet, Avro, etc. You can also understand how to manage metadata across your development teams systems and beyond, with a practical knowledge of tracking data provenance and data lineage.
You'll work to agreed and recognised standards to build solutions using various traditional or big data languages such as: Python, Java, SQL, Scala, PL/SQL (Oracle), SAS Macro Language, etc. whilst using various data tools for example Jupyter, Hue (Hive/Impala), Kibana (Elastic Search), SageMaker (AWS), SAS for example.
Training and development We're committed to developing our people and recognise this will look different for everyone. We use the Government Digital and Data Profession Capability Framework as a compass to provide guidance and tools to support you, and which recognises and rewards specialist expertise.
There'll be opportunities to gain qualifications and pursue specialist pathways, as well as undertaking tailored training, coaching, and mentoring. Individuals can also attend hackathons and have up to 20% innovation and personal development time.
As you develop, grow, and maintain your technical skills, you can progress through the Government Digital and Data pay points. You'll need to regularly demonstrate and accredit your skills based on the framework. If you don't maintain the required skills, you'll be given the opportunity and support to rectify this, as you could revert to a lower level of capability and pay.
Rewards and benefits You'll receive a starting salary of £66,332 dependent on skills, experience, and location plus other benefits including:
- 25 Days Annual Leave automatically rising to 30 days after 5 years' service, and an additional 10.5 days public and privilege holidays.
- opportunities to be recognised through our employee performance scheme
- dedicated development budget
- interest free season ticket loan
- cycle to work scheme
- facilities such as a gym, restaurant, and on site coffee bars (at some locations)
- paid parental and adoption leave.
We're Disability Confident MI5 is proud to have achieved Leader status within the Department for Work and Pensions' Disability Confident scheme. This is aimed at encouraging employers to think differently about disability and take action to improve how they recruit, retain and develop disabled people. As a Disability Confident Leader, we aim to ensure that a fair and proportionate number of disabled applicants who best meet the essential minimum criteria for this position, will be offered an interview, if it is practical for us to do so. (This is known as the Offer of an Interview.) To secure an interview for this role, the minimum criteria (in order of application process) are:
- you will be required to reach the minimum pass mark for the Organisational Situational Judgement Test which looks at your ability to problem solve. If you meet this criteria, you will be directed to complete an application form
- able to demonstrate your ability in developing data solution (assessed at application form sift)
- can demonstrate an understanding of the principles of modern standards approaches such as continuous integration and delivery . click apply for full job details
About MI5