Senior Data Engineer (Databricks) - Contract Bristol | Office-first, then hybrid Initial 3 months, likely to extend | Up to £550 per day, outside IR35 Help build a new data platform from the ground up. My client is growing its technology team and building a SaaS data platform on Microsoft Azure for enterprise customers. The team is small, based in Bristol and moves fast. They need a hands-on Databricks specialist to stand up the platform, get the first pipelines live and set the patterns that future work will build on. What you'll be doing Standing up Azure Databricks as code: workspaces, Unity Catalog, permissions and environments Building reusable, config-driven frameworks for ingestion, medallion layers, data quality and publishing Delivering pipelines end to end, from source systems through to a modelled gold layer published to Azure SQL Implementing CDC ingestion with full history Setting engineering standards: CI/CD, testing, environment promotion, scheduling and alerting Making the platform production-ready: runbooks, backfill and replay, recovery, governance, lineage and cost control Supporting go-lives and stabilising pipelines after launch Using AI-assisted delivery with strong human review Coaching the engineers around you so they can run pipelines confidently without you What you'll bring 6+ years in data engineering, including 3+ years running production pipelines on Databricks Experience taking a Databricks platform from an empty workspace to reliable production Deep, current knowledge of Unity Catalog, Asset Bundles, Delta Lake (MERGE and incremental loads), PySpark, Spark SQL, Jobs and serverless compute Strong ingestion experience: Lakeflow Connect or equivalent CDC from SQL Server, Auto Loader, file-based sources and SCD Type 2 A track record of building frameworks that other engineers adopted Proven use of AI-assisted development on production work, with good judgement about where the tools fall short A preference for working in person, and the ability to be in the Bristol office at the start and regularly after that Eligibility for UK SC clearance Nice to have: Databricks Data Engineer Professional certification, Lakeflow Declarative Pipelines or Lakebase, Terraform, ADLS, Key Vault and Entra ID, data contracts or semantic layers, or current SC clearance. Why this role? You'll shape how the platform is built from day one, and your work will be used long after the contract ends. Interested? Apply now or message me directly for a confidential chat.
10/11/2026
Contractor
Senior Data Engineer (Databricks) - Contract Bristol | Office-first, then hybrid Initial 3 months, likely to extend | Up to £550 per day, outside IR35 Help build a new data platform from the ground up. My client is growing its technology team and building a SaaS data platform on Microsoft Azure for enterprise customers. The team is small, based in Bristol and moves fast. They need a hands-on Databricks specialist to stand up the platform, get the first pipelines live and set the patterns that future work will build on. What you'll be doing Standing up Azure Databricks as code: workspaces, Unity Catalog, permissions and environments Building reusable, config-driven frameworks for ingestion, medallion layers, data quality and publishing Delivering pipelines end to end, from source systems through to a modelled gold layer published to Azure SQL Implementing CDC ingestion with full history Setting engineering standards: CI/CD, testing, environment promotion, scheduling and alerting Making the platform production-ready: runbooks, backfill and replay, recovery, governance, lineage and cost control Supporting go-lives and stabilising pipelines after launch Using AI-assisted delivery with strong human review Coaching the engineers around you so they can run pipelines confidently without you What you'll bring 6+ years in data engineering, including 3+ years running production pipelines on Databricks Experience taking a Databricks platform from an empty workspace to reliable production Deep, current knowledge of Unity Catalog, Asset Bundles, Delta Lake (MERGE and incremental loads), PySpark, Spark SQL, Jobs and serverless compute Strong ingestion experience: Lakeflow Connect or equivalent CDC from SQL Server, Auto Loader, file-based sources and SCD Type 2 A track record of building frameworks that other engineers adopted Proven use of AI-assisted development on production work, with good judgement about where the tools fall short A preference for working in person, and the ability to be in the Bristol office at the start and regularly after that Eligibility for UK SC clearance Nice to have: Databricks Data Engineer Professional certification, Lakeflow Declarative Pipelines or Lakebase, Terraform, ADLS, Key Vault and Entra ID, data contracts or semantic layers, or current SC clearance. Why this role? You'll shape how the platform is built from day one, and your work will be used long after the contract ends. Interested? Apply now or message me directly for a confidential chat.
Data Engineer / SC Cleared 390 - 500 per day (Inside IR35/Umbrella) Duration: 14 Months (60 Weeks) Location: Hybrid from either Newcastle, London, Manchester or Glasgow (2-3 days per week) Start: ASAP We are looking for an experienced Data Engineer to join a high-profile programme, delivering scalable and robust data solutions within a modern cloud-based environment. This is an excellent opportunity to work with cutting-edge technologies including AWS, Kafka, Kubernetes, Java, and Big Data platforms , helping organisations unlock the value of their data. The Role As a Data Engineer, you'll be responsible for designing, developing, and maintaining data solutions that support data generation, collection, processing, and consumption across the organisation. You'll work closely with technical and business stakeholders to build high-performing data pipelines, ensure data quality, and deliver reliable ETL processes. This role requires someone who can provide technical leadership, contribute to key architectural decisions, and collaborate effectively across multiple teams while mentoring junior engineers. Key Responsibilities Design, develop, and maintain scalable data engineering solutions. Build and optimise data pipelines to support large-scale data processing. Implement ETL processes to migrate, transform, and deploy data across multiple platforms. Ensure the quality, accuracy, and reliability of data across the estate. Collaborate with cross-functional teams to understand business requirements and deliver effective data solutions. Monitor, troubleshoot, and improve pipeline performance and stability. Contribute to technical design discussions and key project decisions. Act as a subject matter expert within the data engineering capability. Mentor and support junior team members, promoting best practices and knowledge sharing. Stay current with emerging technologies and industry trends to drive continuous improvement. Required Skills & Experience Strong commercial experience in Data Engineering . Expert-level proficiency in Java . Advanced experience with Amazon Web Services (AWS) . Advanced experience with Apache Kafka . Advanced experience with Kubernetes . Experience working with Big Data technologies and engineering practices. Proven experience building and maintaining enterprise-scale data pipelines. Strong understanding of data modelling, ETL frameworks, and data quality management. Experience working within Agile delivery environments. Excellent stakeholder management and communication skills.
10/11/2026
Contractor
Data Engineer / SC Cleared 390 - 500 per day (Inside IR35/Umbrella) Duration: 14 Months (60 Weeks) Location: Hybrid from either Newcastle, London, Manchester or Glasgow (2-3 days per week) Start: ASAP We are looking for an experienced Data Engineer to join a high-profile programme, delivering scalable and robust data solutions within a modern cloud-based environment. This is an excellent opportunity to work with cutting-edge technologies including AWS, Kafka, Kubernetes, Java, and Big Data platforms , helping organisations unlock the value of their data. The Role As a Data Engineer, you'll be responsible for designing, developing, and maintaining data solutions that support data generation, collection, processing, and consumption across the organisation. You'll work closely with technical and business stakeholders to build high-performing data pipelines, ensure data quality, and deliver reliable ETL processes. This role requires someone who can provide technical leadership, contribute to key architectural decisions, and collaborate effectively across multiple teams while mentoring junior engineers. Key Responsibilities Design, develop, and maintain scalable data engineering solutions. Build and optimise data pipelines to support large-scale data processing. Implement ETL processes to migrate, transform, and deploy data across multiple platforms. Ensure the quality, accuracy, and reliability of data across the estate. Collaborate with cross-functional teams to understand business requirements and deliver effective data solutions. Monitor, troubleshoot, and improve pipeline performance and stability. Contribute to technical design discussions and key project decisions. Act as a subject matter expert within the data engineering capability. Mentor and support junior team members, promoting best practices and knowledge sharing. Stay current with emerging technologies and industry trends to drive continuous improvement. Required Skills & Experience Strong commercial experience in Data Engineering . Expert-level proficiency in Java . Advanced experience with Amazon Web Services (AWS) . Advanced experience with Apache Kafka . Advanced experience with Kubernetes . Experience working with Big Data technologies and engineering practices. Proven experience building and maintaining enterprise-scale data pipelines. Strong understanding of data modelling, ETL frameworks, and data quality management. Experience working within Agile delivery environments. Excellent stakeholder management and communication skills.
I'm currently recruiting for an SC Cleared Databricks Data Engineer role with a leading consultancy delivering a major AWS Databricks platform programme. Required skills: Active SC Clearance (must have) Strong experience deploying and administering Databricks on AWS Databricks Unity Catalog, permissions models, access controls and identity management AWS networking, private connectivity and network isolation for Databricks Infrastructure design, platform governance and disaster recovery planning Terraform for infrastructure, access and configuration management GitHub CI/CD pipelines and release management Databricks workspace and account administration Solution architecture and platform advisory experience Knowledge transfer, documentation and stakeholder engagement Location - Remote (UK based candidates only) Daily rate - £450 (Outside IR35) Contract length - 3 months initially (likely extension)
10/11/2026
Contractor
I'm currently recruiting for an SC Cleared Databricks Data Engineer role with a leading consultancy delivering a major AWS Databricks platform programme. Required skills: Active SC Clearance (must have) Strong experience deploying and administering Databricks on AWS Databricks Unity Catalog, permissions models, access controls and identity management AWS networking, private connectivity and network isolation for Databricks Infrastructure design, platform governance and disaster recovery planning Terraform for infrastructure, access and configuration management GitHub CI/CD pipelines and release management Databricks workspace and account administration Solution architecture and platform advisory experience Knowledge transfer, documentation and stakeholder engagement Location - Remote (UK based candidates only) Daily rate - £450 (Outside IR35) Contract length - 3 months initially (likely extension)
Lead Data Engineer (AI & Data Platforms) | Must Be DV Clearable | 5 Days Onsite Hands-on technical leadership building data and AI systems that actually work We're looking for a Lead Data Engineer to join a growing enterprise data and AI capability, playing a critical role in designing, building, and operating production-grade data platforms and AI-enabled solutions. This is not a line-management role . Instead, it is a senior, hands-on technical leadership position for someone with deep data engineering experience who leads through delivery, technical judgement, and engineering excellence rather than hierarchy. You will work across complex environments spanning Legacy platforms, modern cloud services, advanced analytics, machine learning, and emerging AI technologies. The challenge isn't greenfield experimentation; it's building robust production systems, operationalising AI capabilities, improving what exists, and helping teams use data and AI safely, effectively, and at scale. We're particularly interested in engineers who have helped organisations establish the foundations for AI-led development, including data platforms, ML pipelines, governance frameworks, deployment processes, and production support models. What You'll Be Doing You'll act as a senior technical anchor for both data engineering and AI-enablement initiatives, contributing directly while guiding others through example. You will: Design, build, and enhance enterprise-scale data platforms, pipelines, and services Lead complex data engineering work end-to-end, from problem definition and exploration through build, testing, deployment, and operational support Build and maintain cloud-native data platforms that support analytics, machine learning, and AI workloads Develop robust data pipelines supporting both traditional analytics and AI/ML use cases Work directly with technical and non-technical stakeholders to translate operational problems into scalable data and AI solutions Design data models, integration patterns, and storage structures that support long-term maintainability and reusability Help establish engineering frameworks for AI-led development, including environments, CI/CD processes, testing strategies, governance controls, and deployment standards Support the operationalisation of machine learning and AI solutions into production environments Improve data sharing, onboarding of new data sources, and interoperability across teams and systems Evaluate and introduce new tools, frameworks, and technologies where they deliver genuine value Build robust, production-grade solutions rather than proofs of concept Treat metadata, lineage, observability, governance, data quality, and security as first-class engineering concerns Diagnose and resolve complex data and platform issues in challenging enterprise environments Contribute to a supportive engineering culture focused on quality, delivery, and continuous improvement What We're Looking For This role is for a genuine practitioner who enjoys doing the work and raising standards around them. Essential Experience Significant hands-on experience as a Senior or Lead Data Engineer working on complex, enterprise-scale systems Strong Python and Spark skills with evidence of building and maintaining production data pipelines Deep understanding of the full data engineering lifecycle: ingestion, transformation, storage, serving, and reuse Strong experience designing integrations across diverse data sources, platforms, and Legacy environments Experience building and supporting cloud-based data platforms, preferably within AWS environments Proven experience designing scalable data models and architectures Solid understanding of data governance, security, compliance, and operational controls Experience with modern engineering practices including source control, automated testing, CI/CD, infrastructure as code, and deployment automation Ability to communicate effectively with both highly technical and non-technical stakeholders Experience supporting machine learning or AI workloads through production-grade data engineering solutions Highly Desirable Strong AWS data platform experience (S3, Glue, EMR, Redshift, Athena, Lambda, Bedrock, SageMaker or equivalent) Experience operationalising machine learning models into production environments Experience with MLOps, model deployment, model monitoring, model lifecycle management, and automated validation processes Experience supporting Generative AI solutions, including RAG architectures, vector search, LLM integration, AI orchestration, or agentic workflows Databricks experience, including Unity Catalog, MLflow, Delta Lake, Lakehouse architecture, and AI tooling Experience working within secure, regulated, or government environments Exposure to large-scale enterprise data and AI platforms If you're passionate about building enterprise-scale data platforms, enabling AI adoption, and solving genuinely difficult engineering challenges, we'd love to hear from you. Apply now. Guidant, Carbon60, Lorien & SRG - The Vertage Group Portfolio are acting as an Employment Business in relation to this vacancy.
10/11/2026
Contractor
Lead Data Engineer (AI & Data Platforms) | Must Be DV Clearable | 5 Days Onsite Hands-on technical leadership building data and AI systems that actually work We're looking for a Lead Data Engineer to join a growing enterprise data and AI capability, playing a critical role in designing, building, and operating production-grade data platforms and AI-enabled solutions. This is not a line-management role . Instead, it is a senior, hands-on technical leadership position for someone with deep data engineering experience who leads through delivery, technical judgement, and engineering excellence rather than hierarchy. You will work across complex environments spanning Legacy platforms, modern cloud services, advanced analytics, machine learning, and emerging AI technologies. The challenge isn't greenfield experimentation; it's building robust production systems, operationalising AI capabilities, improving what exists, and helping teams use data and AI safely, effectively, and at scale. We're particularly interested in engineers who have helped organisations establish the foundations for AI-led development, including data platforms, ML pipelines, governance frameworks, deployment processes, and production support models. What You'll Be Doing You'll act as a senior technical anchor for both data engineering and AI-enablement initiatives, contributing directly while guiding others through example. You will: Design, build, and enhance enterprise-scale data platforms, pipelines, and services Lead complex data engineering work end-to-end, from problem definition and exploration through build, testing, deployment, and operational support Build and maintain cloud-native data platforms that support analytics, machine learning, and AI workloads Develop robust data pipelines supporting both traditional analytics and AI/ML use cases Work directly with technical and non-technical stakeholders to translate operational problems into scalable data and AI solutions Design data models, integration patterns, and storage structures that support long-term maintainability and reusability Help establish engineering frameworks for AI-led development, including environments, CI/CD processes, testing strategies, governance controls, and deployment standards Support the operationalisation of machine learning and AI solutions into production environments Improve data sharing, onboarding of new data sources, and interoperability across teams and systems Evaluate and introduce new tools, frameworks, and technologies where they deliver genuine value Build robust, production-grade solutions rather than proofs of concept Treat metadata, lineage, observability, governance, data quality, and security as first-class engineering concerns Diagnose and resolve complex data and platform issues in challenging enterprise environments Contribute to a supportive engineering culture focused on quality, delivery, and continuous improvement What We're Looking For This role is for a genuine practitioner who enjoys doing the work and raising standards around them. Essential Experience Significant hands-on experience as a Senior or Lead Data Engineer working on complex, enterprise-scale systems Strong Python and Spark skills with evidence of building and maintaining production data pipelines Deep understanding of the full data engineering lifecycle: ingestion, transformation, storage, serving, and reuse Strong experience designing integrations across diverse data sources, platforms, and Legacy environments Experience building and supporting cloud-based data platforms, preferably within AWS environments Proven experience designing scalable data models and architectures Solid understanding of data governance, security, compliance, and operational controls Experience with modern engineering practices including source control, automated testing, CI/CD, infrastructure as code, and deployment automation Ability to communicate effectively with both highly technical and non-technical stakeholders Experience supporting machine learning or AI workloads through production-grade data engineering solutions Highly Desirable Strong AWS data platform experience (S3, Glue, EMR, Redshift, Athena, Lambda, Bedrock, SageMaker or equivalent) Experience operationalising machine learning models into production environments Experience with MLOps, model deployment, model monitoring, model lifecycle management, and automated validation processes Experience supporting Generative AI solutions, including RAG architectures, vector search, LLM integration, AI orchestration, or agentic workflows Databricks experience, including Unity Catalog, MLflow, Delta Lake, Lakehouse architecture, and AI tooling Experience working within secure, regulated, or government environments Exposure to large-scale enterprise data and AI platforms If you're passionate about building enterprise-scale data platforms, enabling AI adoption, and solving genuinely difficult engineering challenges, we'd love to hear from you. Apply now. Guidant, Carbon60, Lorien & SRG - The Vertage Group Portfolio are acting as an Employment Business in relation to this vacancy.
Databricks Data Engineer (SC Cleared) Contract: 3 months Rate: £400 per day Location: Fully Remote (UK) Clearance: Active SC Clearance Required We're supporting a secure cloud data programme and are looking for an experienced Databricks Data Engineer to join a high-performing team on an initial 3-month contract. This role requires strong hands-on Databricks experience, with a particular focus on platform administration, security, governance, infrastructure automation and AWS-hosted Databricks environments. You'll work closely with engineering, platform and delivery teams to help maintain, optimise and enhance a critical data platform. Key Responsibilities Develop, maintain and optimise data solutions within Databricks. Administer Databricks workspaces and account-level configurations. Manage permissions, governance and security controls using Unity Catalog. Support identity and access management, including user, group and role-based access controls. Work with AWS networking teams to ensure secure connectivity and platform access. Support disaster recovery planning, testing and documentation. Build and maintain Infrastructure as Code using Terraform. Develop and support GitHub CI/CD pipelines for platform and engineering deployments. Troubleshoot platform, deployment and access-related issues. Produce and maintain technical documentation and operational runbooks. Support knowledge transfer and platform enablement activities with client teams. Essential Skills Strong commercial experience as a Data Engineer working with Databricks. Experience administering Databricks environments at workspace and account level. Strong understanding of Unity Catalog and Databricks security models. Experience with Databricks on AWS. Terraform and Infrastructure as Code experience. Experience building and maintaining GitHub CI/CD pipelines. Knowledge of cloud networking, access controls and platform governance. Strong troubleshooting and problem-solving skills. Excellent documentation and stakeholder communication skills. Active SC Clearance. Desirable Experience within government, defence or other regulated environments. Azure Databricks exposure. Experience supporting platform migrations or cloud transformation programmes. Knowledge of disaster recovery and business continuity planning.
10/11/2026
Contractor
Databricks Data Engineer (SC Cleared) Contract: 3 months Rate: £400 per day Location: Fully Remote (UK) Clearance: Active SC Clearance Required We're supporting a secure cloud data programme and are looking for an experienced Databricks Data Engineer to join a high-performing team on an initial 3-month contract. This role requires strong hands-on Databricks experience, with a particular focus on platform administration, security, governance, infrastructure automation and AWS-hosted Databricks environments. You'll work closely with engineering, platform and delivery teams to help maintain, optimise and enhance a critical data platform. Key Responsibilities Develop, maintain and optimise data solutions within Databricks. Administer Databricks workspaces and account-level configurations. Manage permissions, governance and security controls using Unity Catalog. Support identity and access management, including user, group and role-based access controls. Work with AWS networking teams to ensure secure connectivity and platform access. Support disaster recovery planning, testing and documentation. Build and maintain Infrastructure as Code using Terraform. Develop and support GitHub CI/CD pipelines for platform and engineering deployments. Troubleshoot platform, deployment and access-related issues. Produce and maintain technical documentation and operational runbooks. Support knowledge transfer and platform enablement activities with client teams. Essential Skills Strong commercial experience as a Data Engineer working with Databricks. Experience administering Databricks environments at workspace and account level. Strong understanding of Unity Catalog and Databricks security models. Experience with Databricks on AWS. Terraform and Infrastructure as Code experience. Experience building and maintaining GitHub CI/CD pipelines. Knowledge of cloud networking, access controls and platform governance. Strong troubleshooting and problem-solving skills. Excellent documentation and stakeholder communication skills. Active SC Clearance. Desirable Experience within government, defence or other regulated environments. Azure Databricks exposure. Experience supporting platform migrations or cloud transformation programmes. Knowledge of disaster recovery and business continuity planning.
Senior Data Engineer Position Description Drive measurable change by shaping and delivering large-scale data products and platforms using Palantir Foundry and cloud-native tooling. In this role you'll engineer scalable pipelines, transform complex datasets into operational decision-making assets, and partner with clients to unlock efficiency and insight across critical programmes. You'll join multi-disciplinary teams where pragmatic ownership, creative solutioning and strong technical craft translate directly into business outcomes-building reusable patterns, improving observability and accelerating client roadmaps for modern data adoption. CGI was recognised in the Sunday Times Best Places to Work List 2025 and has been named a UK 'Best Employer' by the Financial Times. We offer a competitive salary, excellent pension, private healthcare, plus a share scheme (3.5% + 3.5% matching) which makes you a CGI Partner not just an employee. We are committed to inclusivity, building a genuinely diverse community of tech talent and inspiring everyone to pursue careers in our sector, including our Armed Forces, and are proud to hold a Gold Award in recognition of our support of the Armed Forces Corporate Covenant. Join us and you'll be part of an open, friendly community of experts. We'll train and support you in taking your career wherever you want it to go. Due to the secure nature of the programme, you will need to hold UK Security Clearance or be eligible to go through this clearance. This is a hybrid position. Your future duties and responsibilities In this role you will design, build and optimise enterprise-scale data pipelines and data products that directly improve client operations and analytics. You will work across Foundry and cloud ecosystems (or quickly gain Foundry expertise), translating stakeholder needs into maintainable, production-grade solutions while championing quality, observability and reusable engineering patterns. You will collaborate closely with architects, product owners and analysts to shape roadmaps and deliver measurable outcomes. Required qualifications to be successful in this role You should have strong commercial experience in data engineering, demonstrating the ability to deliver production-grade solutions on modern platforms. Candidates will be comfortable with large datasets, distributed processing and applying software engineering discipline to data products; Foundry experience is highly desirable but strong Python/PySpark engineers who can learn Foundry quickly are encouraged to apply. • Extensive experience with Python and PySpark / Apache Spark. • Proven track record designing and delivering scalable data pipelines and data products. • Strong SQL and data modelling skills. • Experience with modern data architectures and cloud data services (AWS, Azure or GCP). • Familiarity with version control, automated testing and CI/CD for data platforms. • Experience integrating data from multiple structured and unstructured sources. • Strong problem-solving ability and experience working in delivery-focused or client-facing environments. • (Desirable) Hands-on Palantir Foundry experience or demonstrable ability to upskill quickly in new platforms. Together, as owners, let's turn meaningful insights into action. Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you'll reach your full potential because You are invited to be an owner from day 1 as we work together to bring our Dream to life. That's why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company's strategy and direction. Your work creates value. You'll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You'll shape your career by joining a company built to grow and last. You'll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons. That same commitment to fairness extends to how we use technology. To support our recruitment team, AI tools may be used to help assess applications though they never replace human judgement. All hiring decisions remain entirely in the hands of our recruitment professionals. Come join our team-one of the largest IT and business consulting services firms in the world.
10/11/2026
Full time
Senior Data Engineer Position Description Drive measurable change by shaping and delivering large-scale data products and platforms using Palantir Foundry and cloud-native tooling. In this role you'll engineer scalable pipelines, transform complex datasets into operational decision-making assets, and partner with clients to unlock efficiency and insight across critical programmes. You'll join multi-disciplinary teams where pragmatic ownership, creative solutioning and strong technical craft translate directly into business outcomes-building reusable patterns, improving observability and accelerating client roadmaps for modern data adoption. CGI was recognised in the Sunday Times Best Places to Work List 2025 and has been named a UK 'Best Employer' by the Financial Times. We offer a competitive salary, excellent pension, private healthcare, plus a share scheme (3.5% + 3.5% matching) which makes you a CGI Partner not just an employee. We are committed to inclusivity, building a genuinely diverse community of tech talent and inspiring everyone to pursue careers in our sector, including our Armed Forces, and are proud to hold a Gold Award in recognition of our support of the Armed Forces Corporate Covenant. Join us and you'll be part of an open, friendly community of experts. We'll train and support you in taking your career wherever you want it to go. Due to the secure nature of the programme, you will need to hold UK Security Clearance or be eligible to go through this clearance. This is a hybrid position. Your future duties and responsibilities In this role you will design, build and optimise enterprise-scale data pipelines and data products that directly improve client operations and analytics. You will work across Foundry and cloud ecosystems (or quickly gain Foundry expertise), translating stakeholder needs into maintainable, production-grade solutions while championing quality, observability and reusable engineering patterns. You will collaborate closely with architects, product owners and analysts to shape roadmaps and deliver measurable outcomes. Required qualifications to be successful in this role You should have strong commercial experience in data engineering, demonstrating the ability to deliver production-grade solutions on modern platforms. Candidates will be comfortable with large datasets, distributed processing and applying software engineering discipline to data products; Foundry experience is highly desirable but strong Python/PySpark engineers who can learn Foundry quickly are encouraged to apply. • Extensive experience with Python and PySpark / Apache Spark. • Proven track record designing and delivering scalable data pipelines and data products. • Strong SQL and data modelling skills. • Experience with modern data architectures and cloud data services (AWS, Azure or GCP). • Familiarity with version control, automated testing and CI/CD for data platforms. • Experience integrating data from multiple structured and unstructured sources. • Strong problem-solving ability and experience working in delivery-focused or client-facing environments. • (Desirable) Hands-on Palantir Foundry experience or demonstrable ability to upskill quickly in new platforms. Together, as owners, let's turn meaningful insights into action. Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you'll reach your full potential because You are invited to be an owner from day 1 as we work together to bring our Dream to life. That's why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company's strategy and direction. Your work creates value. You'll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You'll shape your career by joining a company built to grow and last. You'll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons. That same commitment to fairness extends to how we use technology. To support our recruitment team, AI tools may be used to help assess applications though they never replace human judgement. All hiring decisions remain entirely in the hands of our recruitment professionals. Come join our team-one of the largest IT and business consulting services firms in the world.
Senior Data Engineer Position Description At CGI, we're helping clients unlock the full value of their data by building modern, scalable platforms that power better decisions and drive meaningful outcomes. As a Senior Data Engineer, you'll play a pivotal role in developing and enhancing enterprise-scale data solutions, working with cutting-edge Azure technologies in a collaborative DevOps environment. You'll partner with clients and multidisciplinary teams to deliver resilient, high-quality data pipelines while helping shape best practices, mentoring colleagues and contributing to innovative solutions that make a lasting impact. CGI was recognised in the Sunday Times Best Places to Work List 2025 and has been named a UK 'Best Employer' by the Financial Times. We offer a competitive salary, excellent pension, private healthcare, plus a share scheme (3.5% + 3.5% matching) which makes you a CGI Partner not just an employee. We are committed to inclusivity, building a genuinely diverse community of tech talent and inspiring everyone to pursue careers in our sector, including our Armed Forces, and are proud to hold a Gold Award in recognition of our support of the Armed Forces Corporate Covenant. Join us and you'll be part of an open, friendly community of experts. We'll train and support you in taking your career wherever you want it to go. Due to the secure nature of the programme, you will need to hold UK Security Clearance or be eligible to go through this clearance. This is a hybrid position with some infrequent (expensed) travel to workshops at various locations, usually Birmingham, London, Manchester, Leeds. Your future duties and responsibilities In this role, you will join an established team responsible for supporting, maintaining and continuously enhancing a large-scale enterprise data platform. Working closely with CGI colleagues and client stakeholders, you'll design and build scalable data pipelines that enable trusted, high-quality data to support business-critical reporting and analytics. You'll contribute throughout the data engineering lifecycle, ensuring solutions are robust, efficient and aligned with architectural standards and modern engineering practices. You'll also collaborate with Data Architects and Data Analysts to deliver well-designed data solutions, improve platform performance and embed engineering best practice. As a senior member of the team, you'll have opportunities to mentor junior colleagues, contribute to continuous improvement initiatives and help drive the adoption of modern data engineering and DevOps approaches. Key responsibilities • Design & build scalable data pipelines using Azure technologies • Develop & maintain Databricks notebooks and data engineering solutions • Create & optimise SQL and Python-based data transformation processes • Implement & manage Azure Data Factory pipelines and orchestration • Collaborate & align with Data Architects on solution design and best practices • Support & enhance data modelling activities with Data Analysts • Monitor & improve data quality, lineage and platform reliability • Troubleshoot & resolve data pipeline issues across development and production environments • Apply & promote Agile delivery and DevOps engineering practices • Maintain & document technical solutions and data integration processes • Coach & mentor junior team members, supporting capability growth Required qualifications to be successful in this role You'll be an experienced Data Engineer with a strong background in building cloud-based data platforms using Microsoft Azure technologies. You'll enjoy solving complex technical challenges, collaborating across multidisciplinary teams and delivering high-quality data solutions that create measurable value for clients. A proactive approach, strong engineering mindset and commitment to continuous improvement will help you succeed in this role. Essential qualifications • Strong experience as a Data Engineer within Azure-based environments • Expertise in Azure Databricks • Strong SQL development and optimisation skills • Experience with Azure Data Factory • Proficiency in Python for data engineering and automation • Experience working within Agile delivery teams • Knowledge of Azure DevOps and modern DevOps practices • Experience designing and maintaining scalable data pipelines • Strong understanding of data transformation, integration and modelling principles • Excellent problem-solving and stakeholder communication skills Desirable experience • Git version control • Azure Blob Storage and Azure Synapse Analytics • CI/CD implementation for data engineering • Data quality, testing and observability tooling • Experience with Triplestore technologies Together, as owners, let's turn meaningful insights into action. Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you'll reach your full potential because You are invited to be an owner from day 1 as we work together to bring our Dream to life. That's why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company's strategy and direction. Your work creates value. You'll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You'll shape your career by joining a company built to grow and last. You'll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons. That same commitment to fairness extends to how we use technology. To support our recruitment team, AI tools may be used to help assess applications though they never replace human judgement. All hiring decisions remain entirely in the hands of our recruitment professionals. Come join our team-one of the largest IT and business consulting services firms in the world.
10/11/2026
Full time
Senior Data Engineer Position Description At CGI, we're helping clients unlock the full value of their data by building modern, scalable platforms that power better decisions and drive meaningful outcomes. As a Senior Data Engineer, you'll play a pivotal role in developing and enhancing enterprise-scale data solutions, working with cutting-edge Azure technologies in a collaborative DevOps environment. You'll partner with clients and multidisciplinary teams to deliver resilient, high-quality data pipelines while helping shape best practices, mentoring colleagues and contributing to innovative solutions that make a lasting impact. CGI was recognised in the Sunday Times Best Places to Work List 2025 and has been named a UK 'Best Employer' by the Financial Times. We offer a competitive salary, excellent pension, private healthcare, plus a share scheme (3.5% + 3.5% matching) which makes you a CGI Partner not just an employee. We are committed to inclusivity, building a genuinely diverse community of tech talent and inspiring everyone to pursue careers in our sector, including our Armed Forces, and are proud to hold a Gold Award in recognition of our support of the Armed Forces Corporate Covenant. Join us and you'll be part of an open, friendly community of experts. We'll train and support you in taking your career wherever you want it to go. Due to the secure nature of the programme, you will need to hold UK Security Clearance or be eligible to go through this clearance. This is a hybrid position with some infrequent (expensed) travel to workshops at various locations, usually Birmingham, London, Manchester, Leeds. Your future duties and responsibilities In this role, you will join an established team responsible for supporting, maintaining and continuously enhancing a large-scale enterprise data platform. Working closely with CGI colleagues and client stakeholders, you'll design and build scalable data pipelines that enable trusted, high-quality data to support business-critical reporting and analytics. You'll contribute throughout the data engineering lifecycle, ensuring solutions are robust, efficient and aligned with architectural standards and modern engineering practices. You'll also collaborate with Data Architects and Data Analysts to deliver well-designed data solutions, improve platform performance and embed engineering best practice. As a senior member of the team, you'll have opportunities to mentor junior colleagues, contribute to continuous improvement initiatives and help drive the adoption of modern data engineering and DevOps approaches. Key responsibilities • Design & build scalable data pipelines using Azure technologies • Develop & maintain Databricks notebooks and data engineering solutions • Create & optimise SQL and Python-based data transformation processes • Implement & manage Azure Data Factory pipelines and orchestration • Collaborate & align with Data Architects on solution design and best practices • Support & enhance data modelling activities with Data Analysts • Monitor & improve data quality, lineage and platform reliability • Troubleshoot & resolve data pipeline issues across development and production environments • Apply & promote Agile delivery and DevOps engineering practices • Maintain & document technical solutions and data integration processes • Coach & mentor junior team members, supporting capability growth Required qualifications to be successful in this role You'll be an experienced Data Engineer with a strong background in building cloud-based data platforms using Microsoft Azure technologies. You'll enjoy solving complex technical challenges, collaborating across multidisciplinary teams and delivering high-quality data solutions that create measurable value for clients. A proactive approach, strong engineering mindset and commitment to continuous improvement will help you succeed in this role. Essential qualifications • Strong experience as a Data Engineer within Azure-based environments • Expertise in Azure Databricks • Strong SQL development and optimisation skills • Experience with Azure Data Factory • Proficiency in Python for data engineering and automation • Experience working within Agile delivery teams • Knowledge of Azure DevOps and modern DevOps practices • Experience designing and maintaining scalable data pipelines • Strong understanding of data transformation, integration and modelling principles • Excellent problem-solving and stakeholder communication skills Desirable experience • Git version control • Azure Blob Storage and Azure Synapse Analytics • CI/CD implementation for data engineering • Data quality, testing and observability tooling • Experience with Triplestore technologies Together, as owners, let's turn meaningful insights into action. Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you'll reach your full potential because You are invited to be an owner from day 1 as we work together to bring our Dream to life. That's why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company's strategy and direction. Your work creates value. You'll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You'll shape your career by joining a company built to grow and last. You'll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons. That same commitment to fairness extends to how we use technology. To support our recruitment team, AI tools may be used to help assess applications though they never replace human judgement. All hiring decisions remain entirely in the hands of our recruitment professionals. Come join our team-one of the largest IT and business consulting services firms in the world.
Data Engineer (Databricks and AWS) Position Description At CGI, we're helping to transform the future of healthcare through the power of data. As a Senior Data Engineer, you'll play a pivotal role in designing, building, and optimising data platforms that underpin critical national services. Working at the heart of our Healthcare team, you'll use your expertise in AWS, Databricks, and Python to deliver high-impact solutions that improve outcomes, enhance decision-making, and drive innovation across the sector. You'll collaborate with experts who share your passion for problem-solving, ownership, and technical excellence-empowered to shape the data foundations of tomorrow. CGI was recognised in the Sunday Times Best Places to Work List 2025 and has been named a UK 'Best Employer' by the Financial Times. We offer a competitive salary, excellent pension, private healthcare, plus a share scheme (3.5% + 3.5% matching) which makes you a CGI Partner not just an employee. We are committed to inclusivity, building a genuinely diverse community of tech talent and inspiring everyone to pursue careers in our sector, including our Armed Forces, and are proud to hold a Gold Award in recognition of our support of the Armed Forces Corporate Covenant. Join us and you'll be part of an open, friendly community of experts. We'll train and support you in taking your career wherever you want it to go. Due to the secure nature of the programme, you will need to hold UK Security Clearance or be eligible to go through this clearance. This is a hybrid position based in Leeds. Your future duties and responsibilities In this role, you will design, build, and maintain data solutions that power some of the UK's most critical healthcare systems. You'll be part of a collaborative engineering team, transforming how data is captured, processed, and used to drive better patient and operational outcomes. Your work will combine technical innovation with practical delivery-enabling data accessibility, quality, and security at scale. You'll take ownership of complex data challenges, partner with architects and analysts to shape technical direction, and continuously refine processes to deliver efficient, sustainable data pipelines. Working within CGI's supportive environment, you'll be encouraged to explore new technologies, share knowledge, and contribute to a culture of excellence and innovation. • Design & Build: Develop and optimise data pipelines using Databricks, Apache Spark, and Python. • Develop & Deliver: Create scalable data solutions on AWS leveraging S3, Glue, Lambda, and related services. • Integrate & Automate: Implement ETL processes and data lake/lakehouse architectures that ensure accuracy and reliability. • Collaborate & Advise: Partner with technical and business stakeholders to translate requirements into effective data solutions. • Secure & Govern: Ensure compliance with data governance, NHS standards, and security frameworks. • Innovate & Improve: Drive continuous improvement across data engineering practices and technologies. Required qualifications to be successful in this role To excel in this role, you'll bring strong data engineering expertise and hands-on experience in cloud-based data solutions, ideally within regulated or complex environments such as healthcare. You'll be confident in both the technical and consultative aspects of data delivery. You must have: • Hands-on commercial expertise with Databricks. You should have: • Proven experience as a Data Engineer working with large, complex datasets. • Hands-on expertise with Apache Spark, and SQL. • Strong proficiency in Python (PySpark preferred). • Experience with AWS cloud services including S3, Glue, Lambda, IAM. • Familiarity with ETL design, data modelling, and data lake/lakehouse concepts. • Understanding of data governance and compliance frameworks. • Experience in the healthcare sector or knowledge of NHS data standards (advantageous). Together, as owners, let's turn meaningful insights into action. Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you'll reach your full potential because You are invited to be an owner from day 1 as we work together to bring our Dream to life. That's why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company's strategy and direction. Your work creates value. You'll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You'll shape your career by joining a company built to grow and last. You'll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons. That same commitment to fairness extends to how we use technology. To support our recruitment team, AI tools may be used to help assess applications though they never replace human judgement. All hiring decisions remain entirely in the hands of our recruitment professionals. Come join our team-one of the largest IT and business consulting services firms in the world.
10/11/2026
Full time
Data Engineer (Databricks and AWS) Position Description At CGI, we're helping to transform the future of healthcare through the power of data. As a Senior Data Engineer, you'll play a pivotal role in designing, building, and optimising data platforms that underpin critical national services. Working at the heart of our Healthcare team, you'll use your expertise in AWS, Databricks, and Python to deliver high-impact solutions that improve outcomes, enhance decision-making, and drive innovation across the sector. You'll collaborate with experts who share your passion for problem-solving, ownership, and technical excellence-empowered to shape the data foundations of tomorrow. CGI was recognised in the Sunday Times Best Places to Work List 2025 and has been named a UK 'Best Employer' by the Financial Times. We offer a competitive salary, excellent pension, private healthcare, plus a share scheme (3.5% + 3.5% matching) which makes you a CGI Partner not just an employee. We are committed to inclusivity, building a genuinely diverse community of tech talent and inspiring everyone to pursue careers in our sector, including our Armed Forces, and are proud to hold a Gold Award in recognition of our support of the Armed Forces Corporate Covenant. Join us and you'll be part of an open, friendly community of experts. We'll train and support you in taking your career wherever you want it to go. Due to the secure nature of the programme, you will need to hold UK Security Clearance or be eligible to go through this clearance. This is a hybrid position based in Leeds. Your future duties and responsibilities In this role, you will design, build, and maintain data solutions that power some of the UK's most critical healthcare systems. You'll be part of a collaborative engineering team, transforming how data is captured, processed, and used to drive better patient and operational outcomes. Your work will combine technical innovation with practical delivery-enabling data accessibility, quality, and security at scale. You'll take ownership of complex data challenges, partner with architects and analysts to shape technical direction, and continuously refine processes to deliver efficient, sustainable data pipelines. Working within CGI's supportive environment, you'll be encouraged to explore new technologies, share knowledge, and contribute to a culture of excellence and innovation. • Design & Build: Develop and optimise data pipelines using Databricks, Apache Spark, and Python. • Develop & Deliver: Create scalable data solutions on AWS leveraging S3, Glue, Lambda, and related services. • Integrate & Automate: Implement ETL processes and data lake/lakehouse architectures that ensure accuracy and reliability. • Collaborate & Advise: Partner with technical and business stakeholders to translate requirements into effective data solutions. • Secure & Govern: Ensure compliance with data governance, NHS standards, and security frameworks. • Innovate & Improve: Drive continuous improvement across data engineering practices and technologies. Required qualifications to be successful in this role To excel in this role, you'll bring strong data engineering expertise and hands-on experience in cloud-based data solutions, ideally within regulated or complex environments such as healthcare. You'll be confident in both the technical and consultative aspects of data delivery. You must have: • Hands-on commercial expertise with Databricks. You should have: • Proven experience as a Data Engineer working with large, complex datasets. • Hands-on expertise with Apache Spark, and SQL. • Strong proficiency in Python (PySpark preferred). • Experience with AWS cloud services including S3, Glue, Lambda, IAM. • Familiarity with ETL design, data modelling, and data lake/lakehouse concepts. • Understanding of data governance and compliance frameworks. • Experience in the healthcare sector or knowledge of NHS data standards (advantageous). Together, as owners, let's turn meaningful insights into action. Life at CGI is rooted in ownership, teamwork, respect and belonging. Here, you'll reach your full potential because You are invited to be an owner from day 1 as we work together to bring our Dream to life. That's why we call ourselves CGI Partners rather than employees. We benefit from our collective success and actively shape our company's strategy and direction. Your work creates value. You'll develop innovative solutions and build relationships with teammates and clients while accessing global capabilities to scale your ideas, embrace new opportunities, and benefit from expansive industry and technology expertise. You'll shape your career by joining a company built to grow and last. You'll be supported by leaders who care about your health and well-being and provide you with opportunities to deepen your skills and broaden your horizons. That same commitment to fairness extends to how we use technology. To support our recruitment team, AI tools may be used to help assess applications though they never replace human judgement. All hiring decisions remain entirely in the hands of our recruitment professionals. Come join our team-one of the largest IT and business consulting services firms in the world.
Location The locations for these roles are Newport and Titchfield (Fareham). This role is aligned to ONS hybrid working principles, which are based on attending the office with purpose. The postholder will be expected to contribute to the organisation's overall 40% office attendance, in line with business and role requirements. The induction process for the role will be conducted in person. Please note only candidates living and working in either England or Wales will be eligible to apply for this role due to the requirements of the apprenticeship providers. About the job Job summary The Office for National Statistics (ONS) is the UK's largest producer of official statistics, covering a range of key economic, social and demographic topics. These include measuring changes in the value of the UK economy, estimating the size, geographic distribution and characteristics of the population, and providing indicators of price inflation, employment, earnings, crime and migration. If you are interested in an exciting career in data and technology and want to become a highly skilled tech professional, we are seeking 5 people who are passionate about data and technology to join our Data Engineering teams on an apprenticeship programme. If you are looking for a career change, or perhaps a school leaver looking at alternatives to full-time class-based learning this may be the perfect opportunity for you. We are looking for candidates who want to make the most of data, love a challenge, and want to make a real difference in the quality of National Statistics. You do not have to be skilled in data engineering as full training will be provided, along with a nationally recognised qualification through an apprenticeship provider. Do you have a desire to learn how to translate technical concepts into non-technical language and to learn what communication is required between internal and external stakeholders? Do you have a desire to learn how to build extract transform load (ETL) pipelines that facilitate the movement and transformation of data across ONS systems? Do you have a desire to learn how to optimise the performance of our queries and data pipelines. Do you have a desire learn how to use industry wide tools such as Python, SQL, Cloudera Data Platform and Google Cloud Platform? Would you like to find opportunities for innovation with new tools and uses of data? Then we'd like to hear from you Please note - We will be holding a candidate information session on Monday 9th November at 12:00 noon. If you're interested in finding out more, please join here - Click here Job description The roles will be working in Data Products, Services and Governance (DPSG), which sits centrally within ONS, provides a cross-ONS capability for data, including data access and preparation, and the provision of data and linked data products.DPSG develops and manages the systems and processes that underpin the distribution of data across ONS and beyond. DPSG provides data to ONS, academia, wider government, and other external users in a safe and controlled way, supporting and encouraging users to maximise data utility, enabling innovation and statistical and research use. As a Data Engineer Apprentice, you will study towards a Level 4 Data Analyst apprenticeship and combine your job and hands-on experience with virtual learning and will include one to-one with tutors and mentors, tutorials, and research projects. You will work with appropriate technologies as a Data Engineer apprentice, you'll gain the technical expertise to take your career in an exciting new direction. By the end of the apprenticeship, you will have achieved an industry-standard qualification and have the skills to progress your career in line with your aspirations. Responsibilities Identify, collect, and migrate data to/from a range of internal and external systems. Use a range of analytical techniques such as data mining, time series forecasting and modelling techniques to identify and predict trends and patterns in data. Assist with data quality checking and cleansing and how to resolve issues. Assist production of data visualisation including performance dashboards and reports. Summarise and present the results of data analysis to a range of stakeholders Application of coding standards. Write clear and well documented code in Python using industry and organisational best practices Business Analysis/ User requirements validation; Matching and linking of datasets using statistical matching techniques; Become familiar with Cloud developments and technology Part-time candidates will be considered, subject to a minimum of 30 hours per week , apprenticeship provider requirements and business needs. What qualfication will I get? The programme will be 2 years and will be delivered by assigned external training providers. The apprenticeship will support apprentices to develop their capability in data engineering. As well as training and support from the external training providers you will have support from a dedicated, highly experienced data engineering mentor. All apprentices who successfully meet the scheme's milestones will receive a Level 4 Data Analyst Diploma. You will gain experience to help you work towards your qualification, as well as attending formal training and other development opportunities that will consist of 20% off the job learning. You will develop other skills that will aid you in the workplace, such as organisation, communication, the ability to work under pressure, teamwork, and lots more, giving you a real and valuable insight into the industry. Person specification Essential Criteria Can manage own workload, and work effectively as part of a team, to ensure deadlines or outputs are met. Can analyse information or evidence from a variety of sources to draw conclusions and inform decision making. Can communicate complex or technical ideas, both in writing and verbally, to stakeholder with differing levels of knowledge or experience. Identifies opportunities to make improvements and gets the support of relevant people to implement the changes. Takes opportunities to develop own skills and share knowledge with colleagues. A passion for technology and a keen technical interest. Standard eligibility criteria for apprenticeships apply: You must reside and work in England or Wales. You must have been at least 16 years old by 31st August 2026. You must not already have an equivalent qualification of the same type, content, and level (or higher). (Example computer science degree) You must not be in fulltime education or undertaking another apprenticeship at the same time. You should hold a GCSE in Maths and English at Grade C or above, an equivalent qualification, or relevant experience in lieu of a qualification may be considered. The training provider makes the final decision on whether funding can be provided per person and the job offer will be made on the condition that funding is available.
10/11/2026
Full time
Location The locations for these roles are Newport and Titchfield (Fareham). This role is aligned to ONS hybrid working principles, which are based on attending the office with purpose. The postholder will be expected to contribute to the organisation's overall 40% office attendance, in line with business and role requirements. The induction process for the role will be conducted in person. Please note only candidates living and working in either England or Wales will be eligible to apply for this role due to the requirements of the apprenticeship providers. About the job Job summary The Office for National Statistics (ONS) is the UK's largest producer of official statistics, covering a range of key economic, social and demographic topics. These include measuring changes in the value of the UK economy, estimating the size, geographic distribution and characteristics of the population, and providing indicators of price inflation, employment, earnings, crime and migration. If you are interested in an exciting career in data and technology and want to become a highly skilled tech professional, we are seeking 5 people who are passionate about data and technology to join our Data Engineering teams on an apprenticeship programme. If you are looking for a career change, or perhaps a school leaver looking at alternatives to full-time class-based learning this may be the perfect opportunity for you. We are looking for candidates who want to make the most of data, love a challenge, and want to make a real difference in the quality of National Statistics. You do not have to be skilled in data engineering as full training will be provided, along with a nationally recognised qualification through an apprenticeship provider. Do you have a desire to learn how to translate technical concepts into non-technical language and to learn what communication is required between internal and external stakeholders? Do you have a desire to learn how to build extract transform load (ETL) pipelines that facilitate the movement and transformation of data across ONS systems? Do you have a desire to learn how to optimise the performance of our queries and data pipelines. Do you have a desire learn how to use industry wide tools such as Python, SQL, Cloudera Data Platform and Google Cloud Platform? Would you like to find opportunities for innovation with new tools and uses of data? Then we'd like to hear from you Please note - We will be holding a candidate information session on Monday 9th November at 12:00 noon. If you're interested in finding out more, please join here - Click here Job description The roles will be working in Data Products, Services and Governance (DPSG), which sits centrally within ONS, provides a cross-ONS capability for data, including data access and preparation, and the provision of data and linked data products.DPSG develops and manages the systems and processes that underpin the distribution of data across ONS and beyond. DPSG provides data to ONS, academia, wider government, and other external users in a safe and controlled way, supporting and encouraging users to maximise data utility, enabling innovation and statistical and research use. As a Data Engineer Apprentice, you will study towards a Level 4 Data Analyst apprenticeship and combine your job and hands-on experience with virtual learning and will include one to-one with tutors and mentors, tutorials, and research projects. You will work with appropriate technologies as a Data Engineer apprentice, you'll gain the technical expertise to take your career in an exciting new direction. By the end of the apprenticeship, you will have achieved an industry-standard qualification and have the skills to progress your career in line with your aspirations. Responsibilities Identify, collect, and migrate data to/from a range of internal and external systems. Use a range of analytical techniques such as data mining, time series forecasting and modelling techniques to identify and predict trends and patterns in data. Assist with data quality checking and cleansing and how to resolve issues. Assist production of data visualisation including performance dashboards and reports. Summarise and present the results of data analysis to a range of stakeholders Application of coding standards. Write clear and well documented code in Python using industry and organisational best practices Business Analysis/ User requirements validation; Matching and linking of datasets using statistical matching techniques; Become familiar with Cloud developments and technology Part-time candidates will be considered, subject to a minimum of 30 hours per week , apprenticeship provider requirements and business needs. What qualfication will I get? The programme will be 2 years and will be delivered by assigned external training providers. The apprenticeship will support apprentices to develop their capability in data engineering. As well as training and support from the external training providers you will have support from a dedicated, highly experienced data engineering mentor. All apprentices who successfully meet the scheme's milestones will receive a Level 4 Data Analyst Diploma. You will gain experience to help you work towards your qualification, as well as attending formal training and other development opportunities that will consist of 20% off the job learning. You will develop other skills that will aid you in the workplace, such as organisation, communication, the ability to work under pressure, teamwork, and lots more, giving you a real and valuable insight into the industry. Person specification Essential Criteria Can manage own workload, and work effectively as part of a team, to ensure deadlines or outputs are met. Can analyse information or evidence from a variety of sources to draw conclusions and inform decision making. Can communicate complex or technical ideas, both in writing and verbally, to stakeholder with differing levels of knowledge or experience. Identifies opportunities to make improvements and gets the support of relevant people to implement the changes. Takes opportunities to develop own skills and share knowledge with colleagues. A passion for technology and a keen technical interest. Standard eligibility criteria for apprenticeships apply: You must reside and work in England or Wales. You must have been at least 16 years old by 31st August 2026. You must not already have an equivalent qualification of the same type, content, and level (or higher). (Example computer science degree) You must not be in fulltime education or undertaking another apprenticeship at the same time. You should hold a GCSE in Maths and English at Grade C or above, an equivalent qualification, or relevant experience in lieu of a qualification may be considered. The training provider makes the final decision on whether funding can be provided per person and the job offer will be made on the condition that funding is available.
Data Engineer (Python SQL AWS) London / WFH to £85k Do you have experience of building data pipelines within finance / Fintech environments? You could be progressing your career in a hands-on Data Engineer role at a scaling Investment Research tech company with great benefits and perks. As a Data Engineer you will be at the intersection of data engineering, database management and platform delivery. You will help build and evolve a modern, scalable AWS cloud based data platform. Your focus will be on designing and maintaining data pipelines, ingestion frameworks and processing workflows, supporting the migration of existing datasets into a unified data architecture. Collaborating closely with the Data Platform Engineer, you will contribute to platform design, data modelling, database optimisation and the onboarding of new data sources. You'll also assist with managing and optimising PostgreSQL and time-series database environments, improving data quality and governance and ensuring platform reliability, performance and scalability. Location / WFH: You'll join colleagues in the London office three days a week with flexibility to work from home the other two days. About you: You have experience as a Data Engineer, ideally working with financial data / market data in a FinTech or financial services environment You have strong SQL and Python coding skills You have AWS experience, including Data Lakes and S3 You have experience of building and supporting large scale data pipelines You have experience with PostgreSQL You have experience with times series data, they use TimescaleDB but other experience with similar platforms is fine You have experience with Orchestration platforms such as Dagster or Airflow You're collaborative with great communication skills What's in it for you: Salary to £85k + bonus X3 weeks per year work anywhere 25 days holiday, increasing to 30 Pension 9% (company contributes 5%, employee contributes 4%) Healthcare for you and your family Cycle to Work scheme Employee Assistance Program Apply now or call to find out more about this Data Engineer (Python SQL AWS) opportunity.
10/11/2026
Full time
Data Engineer (Python SQL AWS) London / WFH to £85k Do you have experience of building data pipelines within finance / Fintech environments? You could be progressing your career in a hands-on Data Engineer role at a scaling Investment Research tech company with great benefits and perks. As a Data Engineer you will be at the intersection of data engineering, database management and platform delivery. You will help build and evolve a modern, scalable AWS cloud based data platform. Your focus will be on designing and maintaining data pipelines, ingestion frameworks and processing workflows, supporting the migration of existing datasets into a unified data architecture. Collaborating closely with the Data Platform Engineer, you will contribute to platform design, data modelling, database optimisation and the onboarding of new data sources. You'll also assist with managing and optimising PostgreSQL and time-series database environments, improving data quality and governance and ensuring platform reliability, performance and scalability. Location / WFH: You'll join colleagues in the London office three days a week with flexibility to work from home the other two days. About you: You have experience as a Data Engineer, ideally working with financial data / market data in a FinTech or financial services environment You have strong SQL and Python coding skills You have AWS experience, including Data Lakes and S3 You have experience of building and supporting large scale data pipelines You have experience with PostgreSQL You have experience with times series data, they use TimescaleDB but other experience with similar platforms is fine You have experience with Orchestration platforms such as Dagster or Airflow You're collaborative with great communication skills What's in it for you: Salary to £85k + bonus X3 weeks per year work anywhere 25 days holiday, increasing to 30 Pension 9% (company contributes 5%, employee contributes 4%) Healthcare for you and your family Cycle to Work scheme Employee Assistance Program Apply now or call to find out more about this Data Engineer (Python SQL AWS) opportunity.
About Scrumconnect Scrumconnect is a multi-award-winning UK public sector digital consultancy, delivering digital transformation, agile delivery and platform engineering across central government. We build user-centred digital services that positively impact the lives of over 40 million UK citizens. With a team of 125+ professionals and a strong commitment to innovation, collaboration and diversity, we work with government departments and public sector organisations to deliver meaningful digital change. Key Technologies: Qlik Sense, Elasticsearch, Kibana, Power BI Role Overview We are looking for an experienced Senior Data Engineer with strong expertise in Business Intelligence (BI), dashboard development, data visualisation and data modelling. The successful candidate will be responsible for gathering and translating business requirements into effective analytical solutions, developing interactive dashboards and providing actionable insights to support business decision-making. You will work closely with business stakeholders and the Analytics Lead, managing priorities and delivering high-quality BI solutions to meet evolving business needs. Key Responsibilities Gather, analyse and translate business and BI requirements into effective analytical solutions. Design, develop and maintain interactive dashboards and data visualisations to provide meaningful business insights. Design and implement scalable data models to support current and future business requirements. Analyse complex datasets, investigate data challenges and identify practical solutions. Use Qlik Sense to develop dashboards, reports and analytical solutions. Leverage Elasticsearch and Kibana for data analysis, visualisation and reporting. Present actionable insights and analytical findings to business stakeholders. Collaborate with stakeholders to understand reporting requirements and deliver appropriate BI solutions. Work closely with the Analytics Lead and adapt to changing priorities and assigned tasks. Ensure BI solutions effectively address business needs and support informed decision-making. Required Skills and Experience Strong experience in the Business Intelligence (BI) domain. Proven ability to gather, interpret and translate business requirements into analytical solutions. Hands-on experience in designing and developing interactive dashboards and data visualisations. Strong expertise in data modelling, including designing and implementing scalable data models. Excellent analytical and problem-solving skills, with the ability to investigate complex data challenges. Proven hands-on experience with Qlik Sense . Practical experience with Elasticsearch and Kibana for data analysis, visualisation and reporting. Strong stakeholder engagement and communication skills. Ability to manage changing priorities and work on tasks assigned by the Analytics Lead. Desirable Skills Experience with Microsoft Power BI . Experience working across multiple BI and analytics tools. Key Technologies Qlik Sense Elasticsearch Kibana Microsoft Power BI (Desirable) Diversity and Inclusion At Scrumconnect, we are committed to building an inclusive and diverse workplace where everyone has equal opportunities to thrive. We strongly encourage applications from candidates of all backgrounds and welcome people with different experiences, perspectives and skills. We are proud that over 40% of our workforce is made up of women, and we continue to champion gender diversity and equality in the technology industry.
10/11/2026
Full time
About Scrumconnect Scrumconnect is a multi-award-winning UK public sector digital consultancy, delivering digital transformation, agile delivery and platform engineering across central government. We build user-centred digital services that positively impact the lives of over 40 million UK citizens. With a team of 125+ professionals and a strong commitment to innovation, collaboration and diversity, we work with government departments and public sector organisations to deliver meaningful digital change. Key Technologies: Qlik Sense, Elasticsearch, Kibana, Power BI Role Overview We are looking for an experienced Senior Data Engineer with strong expertise in Business Intelligence (BI), dashboard development, data visualisation and data modelling. The successful candidate will be responsible for gathering and translating business requirements into effective analytical solutions, developing interactive dashboards and providing actionable insights to support business decision-making. You will work closely with business stakeholders and the Analytics Lead, managing priorities and delivering high-quality BI solutions to meet evolving business needs. Key Responsibilities Gather, analyse and translate business and BI requirements into effective analytical solutions. Design, develop and maintain interactive dashboards and data visualisations to provide meaningful business insights. Design and implement scalable data models to support current and future business requirements. Analyse complex datasets, investigate data challenges and identify practical solutions. Use Qlik Sense to develop dashboards, reports and analytical solutions. Leverage Elasticsearch and Kibana for data analysis, visualisation and reporting. Present actionable insights and analytical findings to business stakeholders. Collaborate with stakeholders to understand reporting requirements and deliver appropriate BI solutions. Work closely with the Analytics Lead and adapt to changing priorities and assigned tasks. Ensure BI solutions effectively address business needs and support informed decision-making. Required Skills and Experience Strong experience in the Business Intelligence (BI) domain. Proven ability to gather, interpret and translate business requirements into analytical solutions. Hands-on experience in designing and developing interactive dashboards and data visualisations. Strong expertise in data modelling, including designing and implementing scalable data models. Excellent analytical and problem-solving skills, with the ability to investigate complex data challenges. Proven hands-on experience with Qlik Sense . Practical experience with Elasticsearch and Kibana for data analysis, visualisation and reporting. Strong stakeholder engagement and communication skills. Ability to manage changing priorities and work on tasks assigned by the Analytics Lead. Desirable Skills Experience with Microsoft Power BI . Experience working across multiple BI and analytics tools. Key Technologies Qlik Sense Elasticsearch Kibana Microsoft Power BI (Desirable) Diversity and Inclusion At Scrumconnect, we are committed to building an inclusive and diverse workplace where everyone has equal opportunities to thrive. We strongly encourage applications from candidates of all backgrounds and welcome people with different experiences, perspectives and skills. We are proud that over 40% of our workforce is made up of women, and we continue to champion gender diversity and equality in the technology industry.
Senior Data Engineer (Azure Databricks) London / WFH to £95k Are you a technologist Data Engineer with experience of working in regulated financial environments? You could be progressing your career at a specialist FinTech and Investment technology business. As a Senior Data Engineer / Architect, you'll take a hands-on lead role in shaping and developing the company's data platform. You'll design and build robust Databricks pipelines, develop reusable data products and medallion architecture and help modernise the data estate, migrating legacy data stores, stored procedures and SSRS reporting onto a governed, scalable platform. Collaborating with the Head of Engineering, platform engineers, data governance colleagues and the existing data team, you'll influence data architecture and engineering standards while remaining actively involved in delivery. You'll also help drive the adoption of AI assisted and agentic development, ensuring AI generated code and data solutions are properly designed, tested and controlled. Location / WFH: You'll join colleagues in the London, City office 2-3 days with flexibility to work from home in a hybrid model. About you: You are an experienced Data Engineer keen to take ownership and drive technical transformation You have a strong knowledge of Databricks combined with a good understanding of SQL and SSRS, experience with Azure Data Lakes nice to have You have experience of working in regulated finance / FinTech environments You have experience with data transformation and modernising reporting You have experience using Agentic AI tools You're collaborative with great communication and problem solving skills What's in it for you: Salary to £95k Private medical and dental insurance plus Virtual GP / Nurse appointments Life Assurance and Income Protection Pension to 15% Holiday allowance starting at 25 days, and rising with service Spend up to 4 weeks working abroad Enhanced family leave (maternity, paternity, adoption) Training, development and study sponsorship Regular social events calendar and wellbeing activities Quarterly recognition and reward events Apply now to find out more about this Senior Data Engineer (Azure Databricks) opportunity.
10/11/2026
Full time
Senior Data Engineer (Azure Databricks) London / WFH to £95k Are you a technologist Data Engineer with experience of working in regulated financial environments? You could be progressing your career at a specialist FinTech and Investment technology business. As a Senior Data Engineer / Architect, you'll take a hands-on lead role in shaping and developing the company's data platform. You'll design and build robust Databricks pipelines, develop reusable data products and medallion architecture and help modernise the data estate, migrating legacy data stores, stored procedures and SSRS reporting onto a governed, scalable platform. Collaborating with the Head of Engineering, platform engineers, data governance colleagues and the existing data team, you'll influence data architecture and engineering standards while remaining actively involved in delivery. You'll also help drive the adoption of AI assisted and agentic development, ensuring AI generated code and data solutions are properly designed, tested and controlled. Location / WFH: You'll join colleagues in the London, City office 2-3 days with flexibility to work from home in a hybrid model. About you: You are an experienced Data Engineer keen to take ownership and drive technical transformation You have a strong knowledge of Databricks combined with a good understanding of SQL and SSRS, experience with Azure Data Lakes nice to have You have experience of working in regulated finance / FinTech environments You have experience with data transformation and modernising reporting You have experience using Agentic AI tools You're collaborative with great communication and problem solving skills What's in it for you: Salary to £95k Private medical and dental insurance plus Virtual GP / Nurse appointments Life Assurance and Income Protection Pension to 15% Holiday allowance starting at 25 days, and rising with service Spend up to 4 weeks working abroad Enhanced family leave (maternity, paternity, adoption) Training, development and study sponsorship Regular social events calendar and wellbeing activities Quarterly recognition and reward events Apply now to find out more about this Senior Data Engineer (Azure Databricks) opportunity.
Lead Data Engineer Build and Lead the Function Salary: Up to 90,000 Location: London Four days per week in the office Ref: J13197 A scaling AI-powered SaaS business is looking for a Lead Data Engineer to build and lead its Data Engineering function, shaping the team, setting the standards and creating the foundations for its next stage of growth. You'll have real ownership and a direct influence on how the business builds and delivers its data products. From improving engineering practices to delivering critical pipelines, you'll see the results of your work in live products and the value they create for customers. Working closely with technical and product leaders, you'll develop engineers, bring structure to delivery and stay hands-on with the most complex technical challenges. It's an opportunity to build a function you can be proud of, with strong engineering principles at its core. What you'll own: Engineering standards, tooling and ways of working. Line management, mentoring and future recruitment. SQL-led transformation pipelines and Python orchestration. Code review, automated testing, Git workflows and CI/CD. Data modelling, deployment controls and production reliability. Integration of AI outputs into production data pipelines. Practical adoption of AI engineering tools, supported by validation and quality controls. What you'll bring: Experience building a Data Engineering function or significantly improving an existing team's practices. Proven line-management experience and a track record of developing engineers. Strong hands-on SQL and Python skills. Solid experience in data modelling, cloud infrastructure and deployment automation. The judgement to balance delivery demands with maintainability and technical quality. This would suit a Lead Data Engineer or hands-on Data Engineering Manager who has raised engineering standards, developed people and made delivery more reliable. If you're ready to bring that experience to a scaling AI business, apply or message me for a confidential conversation. Alternatively, you can refer a friend or colleague through our referral scheme. For each relevant candidate you introduce whom we successfully place, you will be entitled to our gift or voucher scheme. Datatech is one of the UK's leading recruitment agencies specialising in data and analytics and is the host of the critically acclaimed Women in Data event. For more information, visit (url removed) .
10/11/2026
Full time
Lead Data Engineer Build and Lead the Function Salary: Up to 90,000 Location: London Four days per week in the office Ref: J13197 A scaling AI-powered SaaS business is looking for a Lead Data Engineer to build and lead its Data Engineering function, shaping the team, setting the standards and creating the foundations for its next stage of growth. You'll have real ownership and a direct influence on how the business builds and delivers its data products. From improving engineering practices to delivering critical pipelines, you'll see the results of your work in live products and the value they create for customers. Working closely with technical and product leaders, you'll develop engineers, bring structure to delivery and stay hands-on with the most complex technical challenges. It's an opportunity to build a function you can be proud of, with strong engineering principles at its core. What you'll own: Engineering standards, tooling and ways of working. Line management, mentoring and future recruitment. SQL-led transformation pipelines and Python orchestration. Code review, automated testing, Git workflows and CI/CD. Data modelling, deployment controls and production reliability. Integration of AI outputs into production data pipelines. Practical adoption of AI engineering tools, supported by validation and quality controls. What you'll bring: Experience building a Data Engineering function or significantly improving an existing team's practices. Proven line-management experience and a track record of developing engineers. Strong hands-on SQL and Python skills. Solid experience in data modelling, cloud infrastructure and deployment automation. The judgement to balance delivery demands with maintainability and technical quality. This would suit a Lead Data Engineer or hands-on Data Engineering Manager who has raised engineering standards, developed people and made delivery more reliable. If you're ready to bring that experience to a scaling AI business, apply or message me for a confidential conversation. Alternatively, you can refer a friend or colleague through our referral scheme. For each relevant candidate you introduce whom we successfully place, you will be entitled to our gift or voucher scheme. Datatech is one of the UK's leading recruitment agencies specialising in data and analytics and is the host of the critically acclaimed Women in Data event. For more information, visit (url removed) .
Senior Data Engineer SQL / Python / AWS Adria Solutions is recruiting a Senior Data Engineer for a growing business developing a new digital services platform. With established backing, the business is building its data capability from the ground up. As its first dedicated data engineer, you ll have significant input into the architecture, technology choices and standards that underpin reporting, analytics and future AI capabilities. The role Reporting to the CTO, you ll turn platform, transaction and third-party data into trusted datasets for operational insight, financial reporting and product development. This is a hands-on opportunity to own the data platform, build reliable pipelines and establish the foundations for a secure, scalable data environment. Key responsibilities Develop and maintain batch and streaming pipelines across multiple data sources. Build the warehouse or lakehouse architecture and data models supporting reporting and analytics. Establish checks and processes for data accuracy, traceability and governance. Deliver reliable datasets for financial and regulatory reporting. Collaborate with engineers and business stakeholders to make data accessible and useful. Implement appropriate access controls, security and privacy practices to support GDPR and FCA requirements. Prepare the platform for future analytics, AI and machine learning applications. Evaluate tooling and contribute to decisions on the wider data architecture. What we re looking for Commercial experience delivering and supporting production data pipelines and warehouses. Strong SQL skills and proficiency in Python or another relevant programming language. Experience with modern data engineering tools, such as dbt, Airflow or Spark. Experience with cloud data warehouses, such as Snowflake, BigQuery or Amazon Redshift. A solid understanding of data modelling, ETL/ELT and data quality. Knowledge of data governance, security and privacy. Confidence taking ownership and making practical technical decisions in an early-stage environment. Desirable experience Working with banking, payments, financial or regulatory datasets. Streaming technologies such as Kafka and real-time data processing. Building datasets and processes for regulatory reporting. Supporting analytics or machine learning use cases. Experience working within a regulated industry. The package The role offers a £50,000 £75,000 salary, alongside a generous annual leave allowance with bank holidays in addition, a pension scheme and hybrid working in Manchester. You ll also benefit from ongoing learning support, opportunities for career progression and the freedom to shape a new data platform as the business grows. Applicants should have at least two years production experience building data pipelines and warehouses, and the right to work in the UK without requiring visa sponsorship. To find out more, apply with your CV or contact Adria Solutions for a confidential discussion. Senior Data Engineer SQL / Python / AWS
10/11/2026
Full time
Senior Data Engineer SQL / Python / AWS Adria Solutions is recruiting a Senior Data Engineer for a growing business developing a new digital services platform. With established backing, the business is building its data capability from the ground up. As its first dedicated data engineer, you ll have significant input into the architecture, technology choices and standards that underpin reporting, analytics and future AI capabilities. The role Reporting to the CTO, you ll turn platform, transaction and third-party data into trusted datasets for operational insight, financial reporting and product development. This is a hands-on opportunity to own the data platform, build reliable pipelines and establish the foundations for a secure, scalable data environment. Key responsibilities Develop and maintain batch and streaming pipelines across multiple data sources. Build the warehouse or lakehouse architecture and data models supporting reporting and analytics. Establish checks and processes for data accuracy, traceability and governance. Deliver reliable datasets for financial and regulatory reporting. Collaborate with engineers and business stakeholders to make data accessible and useful. Implement appropriate access controls, security and privacy practices to support GDPR and FCA requirements. Prepare the platform for future analytics, AI and machine learning applications. Evaluate tooling and contribute to decisions on the wider data architecture. What we re looking for Commercial experience delivering and supporting production data pipelines and warehouses. Strong SQL skills and proficiency in Python or another relevant programming language. Experience with modern data engineering tools, such as dbt, Airflow or Spark. Experience with cloud data warehouses, such as Snowflake, BigQuery or Amazon Redshift. A solid understanding of data modelling, ETL/ELT and data quality. Knowledge of data governance, security and privacy. Confidence taking ownership and making practical technical decisions in an early-stage environment. Desirable experience Working with banking, payments, financial or regulatory datasets. Streaming technologies such as Kafka and real-time data processing. Building datasets and processes for regulatory reporting. Supporting analytics or machine learning use cases. Experience working within a regulated industry. The package The role offers a £50,000 £75,000 salary, alongside a generous annual leave allowance with bank holidays in addition, a pension scheme and hybrid working in Manchester. You ll also benefit from ongoing learning support, opportunities for career progression and the freedom to shape a new data platform as the business grows. Applicants should have at least two years production experience building data pipelines and warehouses, and the right to work in the UK without requiring visa sponsorship. To find out more, apply with your CV or contact Adria Solutions for a confidential discussion. Senior Data Engineer SQL / Python / AWS
MORELA AI & Engineering Recruitment Data Engineer (RAG & Retrieval) 2-year contract Central London Active SC Outside IR35 Morela is supporting a specialist technology consultancy on a 2-year Data Engineer contract in Central London, delivering on a UK government programme. Outside IR35, rate TBC. We need a genuinely strong all-round data engineer first and foremost, with recent hands-on RAG experience on top. Active SC is essential - eligibility will not be considered. About the Client A specialist technology consultancy embedding engineers into SC-cleared government delivery teams. They focus on long-term programme work and give their engineers real ownership of what they build. The Role You will own the data platform behind the programme, from ingestion and modelling through to the retrieval layer that powers its AI capability. The core of the job is solid data engineering - reliable pipelines, clean models and well-governed data. On top of that you will build the RAG pipelines that turn large volumes of unstructured material into accurate, access-controlled answers. Responsibilities Core data engineering Design, build and maintain robust batch and streaming pipelines (ETL/ELT) across structured and unstructured sources Model data for analytics and downstream use, with clear schemas, lineage and documentation Own data quality, validation, monitoring and alerting so the platform can be trusted Optimise pipelines and storage for performance, cost and scale Apply security, governance and access controls appropriate to an SC environment RAG and retrieval Build ingestion for unstructured content, including document parsing, OCR and metadata enrichment Build production RAG pipelines: chunking, embeddings, hybrid search (vector + BM25), reranking and context assembly Enforce document-level access controls at query time Evaluate retrieval and generation quality with measurable metrics, and iterate Work with self-hosted and open-weight models in secure or air-gapped environments SC clearance: Active SC held, not eligibility Essential Requirements Core data engineering Strong commercial data engineering background, building and running production pipelines end to end Excellent Python and SQL Solid grasp of data modelling, warehousing and lakehouse concepts Hands-on with orchestration (e.g. Airflow, Dagster) and distributed processing (e.g. Spark) Experience with cloud data platforms (AWS, Azure or GCP) and containerised deployment (Docker, Kubernetes) Good engineering practice: version control, CI/CD, testing and infrastructure as code RAG Recent RAG delivery: Hands-on RAG systems built and shipped recently - current work, not an old proof of concept Vector databases (e.g. pgvector, OpenSearch, Qdrant, Weaviate), embeddings, hybrid search and reranking Experience evaluating RAG systems properly (e.g. faithfulness) Streaming platforms such as Kafka Agentic or multi-hop retrieval, GraphRAG, query rewriting vLLM or similar for on-prem model serving LangChain, LlamaIndex, RAGAS or Langfuse Government, defence or national security programmes Desirable Experience Working Environment Central London, SC-cleared environment, embedded in the programme team. Outside IR35, 2-year contract. For immediate consideration email (url removed)
10/11/2026
Contractor
MORELA AI & Engineering Recruitment Data Engineer (RAG & Retrieval) 2-year contract Central London Active SC Outside IR35 Morela is supporting a specialist technology consultancy on a 2-year Data Engineer contract in Central London, delivering on a UK government programme. Outside IR35, rate TBC. We need a genuinely strong all-round data engineer first and foremost, with recent hands-on RAG experience on top. Active SC is essential - eligibility will not be considered. About the Client A specialist technology consultancy embedding engineers into SC-cleared government delivery teams. They focus on long-term programme work and give their engineers real ownership of what they build. The Role You will own the data platform behind the programme, from ingestion and modelling through to the retrieval layer that powers its AI capability. The core of the job is solid data engineering - reliable pipelines, clean models and well-governed data. On top of that you will build the RAG pipelines that turn large volumes of unstructured material into accurate, access-controlled answers. Responsibilities Core data engineering Design, build and maintain robust batch and streaming pipelines (ETL/ELT) across structured and unstructured sources Model data for analytics and downstream use, with clear schemas, lineage and documentation Own data quality, validation, monitoring and alerting so the platform can be trusted Optimise pipelines and storage for performance, cost and scale Apply security, governance and access controls appropriate to an SC environment RAG and retrieval Build ingestion for unstructured content, including document parsing, OCR and metadata enrichment Build production RAG pipelines: chunking, embeddings, hybrid search (vector + BM25), reranking and context assembly Enforce document-level access controls at query time Evaluate retrieval and generation quality with measurable metrics, and iterate Work with self-hosted and open-weight models in secure or air-gapped environments SC clearance: Active SC held, not eligibility Essential Requirements Core data engineering Strong commercial data engineering background, building and running production pipelines end to end Excellent Python and SQL Solid grasp of data modelling, warehousing and lakehouse concepts Hands-on with orchestration (e.g. Airflow, Dagster) and distributed processing (e.g. Spark) Experience with cloud data platforms (AWS, Azure or GCP) and containerised deployment (Docker, Kubernetes) Good engineering practice: version control, CI/CD, testing and infrastructure as code RAG Recent RAG delivery: Hands-on RAG systems built and shipped recently - current work, not an old proof of concept Vector databases (e.g. pgvector, OpenSearch, Qdrant, Weaviate), embeddings, hybrid search and reranking Experience evaluating RAG systems properly (e.g. faithfulness) Streaming platforms such as Kafka Agentic or multi-hop retrieval, GraphRAG, query rewriting vLLM or similar for on-prem model serving LangChain, LlamaIndex, RAGAS or Langfuse Government, defence or national security programmes Desirable Experience Working Environment Central London, SC-cleared environment, embedded in the programme team. Outside IR35, 2-year contract. For immediate consideration email (url removed)
Lead Data Engineer - SC Cleared SR2 are recruiting a Lead Data Engineer to join a multi-year transformation programme underway in government. We're actively recruiting a lead to support a newly formed team focused on data pipelines and Java, Kafka, Kubernetes and AWS. Due to the nature of the roles, active SC clearance is required. Experience required: Active SC clearance Java programming background Kafka Kubernetes AWS This is a multi-year programme with long-term contracts on offer.
10/11/2026
Contractor
Lead Data Engineer - SC Cleared SR2 are recruiting a Lead Data Engineer to join a multi-year transformation programme underway in government. We're actively recruiting a lead to support a newly formed team focused on data pipelines and Java, Kafka, Kubernetes and AWS. Due to the nature of the roles, active SC clearance is required. Experience required: Active SC clearance Java programming background Kafka Kubernetes AWS This is a multi-year programme with long-term contracts on offer.
Data Engineer Telford/Hybrid - 2 days per week onsite £465 per day inside ir35 Active SC clearance ocation: Telford/Hybrid (2 days onsite per week) Rate: Up to £465 per day (Inside IR35) Contract: 6 months Clearance: Active SC required We are seeking an experienced Data Engineer to join a major IT transformation programme, supporting the delivery of a key data engineering project. The role will focus on developing and maintaining data integration solutions using Talend and Oracle RDS , working within an established framework and development environment. Key Responsibilities: Develop Talend jobs, Oracle DDL, SQL and GitLab pipeline enhancements. Investigate and resolve defects identified during testing. Conduct component testing, including test data creation. Support QA teams with testing and debugging activities. Create and execute automated component test packs. Support performance testing and production deployments. Review development artefacts and provide technical guidance to junior engineers. Work closely with the Lead Engineer and wider Agile delivery teams. Essential Skills & Experience: Strong ETL development experience, ideally with Talend. Experience with Pentaho or Informatica will also be considered. Hands-on experience with Oracle RDS databases. Strong Oracle SQL and DDL skills. Familiarity with GitLab and CI/CD pipelines. Experience across development, testing, defect resolution and production support. Candidates must hold active SC clearance. If you have strong ETL and Oracle RDS experience and are looking for your next contract opportunity, please get in touch.
10/11/2026
Contractor
Data Engineer Telford/Hybrid - 2 days per week onsite £465 per day inside ir35 Active SC clearance ocation: Telford/Hybrid (2 days onsite per week) Rate: Up to £465 per day (Inside IR35) Contract: 6 months Clearance: Active SC required We are seeking an experienced Data Engineer to join a major IT transformation programme, supporting the delivery of a key data engineering project. The role will focus on developing and maintaining data integration solutions using Talend and Oracle RDS , working within an established framework and development environment. Key Responsibilities: Develop Talend jobs, Oracle DDL, SQL and GitLab pipeline enhancements. Investigate and resolve defects identified during testing. Conduct component testing, including test data creation. Support QA teams with testing and debugging activities. Create and execute automated component test packs. Support performance testing and production deployments. Review development artefacts and provide technical guidance to junior engineers. Work closely with the Lead Engineer and wider Agile delivery teams. Essential Skills & Experience: Strong ETL development experience, ideally with Talend. Experience with Pentaho or Informatica will also be considered. Hands-on experience with Oracle RDS databases. Strong Oracle SQL and DDL skills. Familiarity with GitLab and CI/CD pipelines. Experience across development, testing, defect resolution and production support. Candidates must hold active SC clearance. If you have strong ETL and Oracle RDS experience and are looking for your next contract opportunity, please get in touch.
Data Engineer Programme - No Experience Required This is a self-funded career programme with a guaranteed job on completion or 100% of your course fees back Train. Certify. Get Hired. Are you looking to start a career in data but don't know where to begin? The demand for Data Analysts has grown by 20% annually, with experienced professionals earning salaries upwards of £58,000. In today's digital world, data is critical to business decision-making, making the role of a Data Analyst indispensable. As skills shortages continue to grow, the demand for qualified entry-level professionals is on the rise. Newto Training's Data Analyst Career Programme is designed to help aspiring professionals gain the qualifications, practical experience and support needed to secure their first role in the industry. Whether you're looking for a complete career change, returning to work, leaving the Armed Forces, or seeking a future-proof career, we'll help you build the skills employers need. Please note: this is a self-funded programme costing around £190 per month How Our Career Programme Works: Over 100 hours of live instructor-led online training delivered by experienced industry professionals Three industry-recognised certifications (exam voucher and exam resit support included): Microsoft Power BI Data Analyst (PL-300) Microsoft Azure Data Fundamentals (DP-900) Microsoft Azure AI Fundamentals (AI-901) Practical, real-world project work designed to help you develop hands-on skills in Excel, SQL, Python and PowerBI Dedicated 1-to-1 tutor support throughout your learning journey CV support, interview preparation, and career coaching Access to our employer network and recruitment partners Starting Salaries Upon successful completion of the programme, we guarantee a starting salary of up to £40,000. Who Is This Programme For? This programme is designed for individuals with little or no previous experience. You may be: Looking for a career change A recent school, college or university leaver Currently working in another industry Leaving the Armed Forces Interested in technology and problem-solving The most important qualities are a willingness to learn, good communication skills and a genuine interest in changing your career. Please note: this is a self-funded programme costing around £190 per month Our Job Guarantee Successfully complete the programme and meet the agreed career support requirements, and we guarantee you'll receive a job offer within 6 months. If not, we'll refund 100% of your course fees. Full T&Cs available. Ready to Start Your New Career? Click 'Apply Now' to speak with an Advisor and take the first step towards a rewarding career in one of the UK's fastest-growing sectors.
10/11/2026
Full time
Data Engineer Programme - No Experience Required This is a self-funded career programme with a guaranteed job on completion or 100% of your course fees back Train. Certify. Get Hired. Are you looking to start a career in data but don't know where to begin? The demand for Data Analysts has grown by 20% annually, with experienced professionals earning salaries upwards of £58,000. In today's digital world, data is critical to business decision-making, making the role of a Data Analyst indispensable. As skills shortages continue to grow, the demand for qualified entry-level professionals is on the rise. Newto Training's Data Analyst Career Programme is designed to help aspiring professionals gain the qualifications, practical experience and support needed to secure their first role in the industry. Whether you're looking for a complete career change, returning to work, leaving the Armed Forces, or seeking a future-proof career, we'll help you build the skills employers need. Please note: this is a self-funded programme costing around £190 per month How Our Career Programme Works: Over 100 hours of live instructor-led online training delivered by experienced industry professionals Three industry-recognised certifications (exam voucher and exam resit support included): Microsoft Power BI Data Analyst (PL-300) Microsoft Azure Data Fundamentals (DP-900) Microsoft Azure AI Fundamentals (AI-901) Practical, real-world project work designed to help you develop hands-on skills in Excel, SQL, Python and PowerBI Dedicated 1-to-1 tutor support throughout your learning journey CV support, interview preparation, and career coaching Access to our employer network and recruitment partners Starting Salaries Upon successful completion of the programme, we guarantee a starting salary of up to £40,000. Who Is This Programme For? This programme is designed for individuals with little or no previous experience. You may be: Looking for a career change A recent school, college or university leaver Currently working in another industry Leaving the Armed Forces Interested in technology and problem-solving The most important qualities are a willingness to learn, good communication skills and a genuine interest in changing your career. Please note: this is a self-funded programme costing around £190 per month Our Job Guarantee Successfully complete the programme and meet the agreed career support requirements, and we guarantee you'll receive a job offer within 6 months. If not, we'll refund 100% of your course fees. Full T&Cs available. Ready to Start Your New Career? Click 'Apply Now' to speak with an Advisor and take the first step towards a rewarding career in one of the UK's fastest-growing sectors.
Data Engineer Programme - No Experience Required This is a self-funded career programme with a guaranteed job on completion or 100% of your course fees back Train. Certify. Get Hired. Are you looking to start a career in data but don't know where to begin? The demand for Data Analysts has grown by 20% annually, with experienced professionals earning salaries upwards of £58,000. In today's digital world, data is critical to business decision-making, making the role of a Data Analyst indispensable. As skills shortages continue to grow, the demand for qualified entry-level professionals is on the rise. Newto Training's Data Analyst Career Programme is designed to help aspiring professionals gain the qualifications, practical experience and support needed to secure their first role in the industry. Whether you're looking for a complete career change, returning to work, leaving the Armed Forces, or seeking a future-proof career, we'll help you build the skills employers need. Please note: this is a self-funded programme costing around £190 per month How Our Career Programme Works: Over 100 hours of live instructor-led online training delivered by experienced industry professionals Three industry-recognised certifications (exam voucher and exam resit support included): Microsoft Power BI Data Analyst (PL-300) Microsoft Azure Data Fundamentals (DP-900) Microsoft Azure AI Fundamentals (AI-901) Practical, real-world project work designed to help you develop hands-on skills in Excel, SQL, Python and PowerBI Dedicated 1-to-1 tutor support throughout your learning journey CV support, interview preparation, and career coaching Access to our employer network and recruitment partners Starting Salaries Upon successful completion of the programme, we guarantee a starting salary of up to £40,000. Who Is This Programme For? This programme is designed for individuals with little or no previous experience. You may be: Looking for a career change A recent school, college or university leaver Currently working in another industry Leaving the Armed Forces Interested in technology and problem-solving The most important qualities are a willingness to learn, good communication skills and a genuine interest in changing your career. Please note: this is a self-funded programme costing around £190 per month Our Job Guarantee Successfully complete the programme and meet the agreed career support requirements, and we guarantee you'll receive a job offer within 6 months. If not, we'll refund 100% of your course fees. Full T&Cs available. Ready to Start Your New Career? Click 'Apply Now' to speak with an Advisor and take the first step towards a rewarding career in one of the UK's fastest-growing sectors.
10/11/2026
Full time
Data Engineer Programme - No Experience Required This is a self-funded career programme with a guaranteed job on completion or 100% of your course fees back Train. Certify. Get Hired. Are you looking to start a career in data but don't know where to begin? The demand for Data Analysts has grown by 20% annually, with experienced professionals earning salaries upwards of £58,000. In today's digital world, data is critical to business decision-making, making the role of a Data Analyst indispensable. As skills shortages continue to grow, the demand for qualified entry-level professionals is on the rise. Newto Training's Data Analyst Career Programme is designed to help aspiring professionals gain the qualifications, practical experience and support needed to secure their first role in the industry. Whether you're looking for a complete career change, returning to work, leaving the Armed Forces, or seeking a future-proof career, we'll help you build the skills employers need. Please note: this is a self-funded programme costing around £190 per month How Our Career Programme Works: Over 100 hours of live instructor-led online training delivered by experienced industry professionals Three industry-recognised certifications (exam voucher and exam resit support included): Microsoft Power BI Data Analyst (PL-300) Microsoft Azure Data Fundamentals (DP-900) Microsoft Azure AI Fundamentals (AI-901) Practical, real-world project work designed to help you develop hands-on skills in Excel, SQL, Python and PowerBI Dedicated 1-to-1 tutor support throughout your learning journey CV support, interview preparation, and career coaching Access to our employer network and recruitment partners Starting Salaries Upon successful completion of the programme, we guarantee a starting salary of up to £40,000. Who Is This Programme For? This programme is designed for individuals with little or no previous experience. You may be: Looking for a career change A recent school, college or university leaver Currently working in another industry Leaving the Armed Forces Interested in technology and problem-solving The most important qualities are a willingness to learn, good communication skills and a genuine interest in changing your career. Please note: this is a self-funded programme costing around £190 per month Our Job Guarantee Successfully complete the programme and meet the agreed career support requirements, and we guarantee you'll receive a job offer within 6 months. If not, we'll refund 100% of your course fees. Full T&Cs available. Ready to Start Your New Career? Click 'Apply Now' to speak with an Advisor and take the first step towards a rewarding career in one of the UK's fastest-growing sectors.
Data Engineer Programme - No Experience Required This is a self-funded career programme with a guaranteed job on completion or 100% of your course fees back Train. Certify. Get Hired. Are you looking to start a career in data but don't know where to begin? The demand for Data Analysts has grown by 20% annually, with experienced professionals earning salaries upwards of £58,000. In today's digital world, data is critical to business decision-making, making the role of a Data Analyst indispensable. As skills shortages continue to grow, the demand for qualified entry-level professionals is on the rise. Newto Training's Data Analyst Career Programme is designed to help aspiring professionals gain the qualifications, practical experience and support needed to secure their first role in the industry. Whether you're looking for a complete career change, returning to work, leaving the Armed Forces, or seeking a future-proof career, we'll help you build the skills employers need. Please note: this is a self-funded programme costing around £190 per month How Our Career Programme Works: Over 100 hours of live instructor-led online training delivered by experienced industry professionals Three industry-recognised certifications (exam voucher and exam resit support included): Microsoft Power BI Data Analyst (PL-300) Microsoft Azure Data Fundamentals (DP-900) Microsoft Azure AI Fundamentals (AI-901) Practical, real-world project work designed to help you develop hands-on skills in Excel, SQL, Python and PowerBI Dedicated 1-to-1 tutor support throughout your learning journey CV support, interview preparation, and career coaching Access to our employer network and recruitment partners Starting Salaries Upon successful completion of the programme, we guarantee a starting salary of up to £40,000. Who Is This Programme For? This programme is designed for individuals with little or no previous experience. You may be: Looking for a career change A recent school, college or university leaver Currently working in another industry Leaving the Armed Forces Interested in technology and problem-solving The most important qualities are a willingness to learn, good communication skills and a genuine interest in changing your career. Please note: this is a self-funded programme costing around £190 per month Our Job Guarantee Successfully complete the programme and meet the agreed career support requirements, and we guarantee you'll receive a job offer within 6 months. If not, we'll refund 100% of your course fees. Full T&Cs available. Ready to Start Your New Career? Click 'Apply Now' to speak with an Advisor and take the first step towards a rewarding career in one of the UK's fastest-growing sectors.
10/11/2026
Full time
Data Engineer Programme - No Experience Required This is a self-funded career programme with a guaranteed job on completion or 100% of your course fees back Train. Certify. Get Hired. Are you looking to start a career in data but don't know where to begin? The demand for Data Analysts has grown by 20% annually, with experienced professionals earning salaries upwards of £58,000. In today's digital world, data is critical to business decision-making, making the role of a Data Analyst indispensable. As skills shortages continue to grow, the demand for qualified entry-level professionals is on the rise. Newto Training's Data Analyst Career Programme is designed to help aspiring professionals gain the qualifications, practical experience and support needed to secure their first role in the industry. Whether you're looking for a complete career change, returning to work, leaving the Armed Forces, or seeking a future-proof career, we'll help you build the skills employers need. Please note: this is a self-funded programme costing around £190 per month How Our Career Programme Works: Over 100 hours of live instructor-led online training delivered by experienced industry professionals Three industry-recognised certifications (exam voucher and exam resit support included): Microsoft Power BI Data Analyst (PL-300) Microsoft Azure Data Fundamentals (DP-900) Microsoft Azure AI Fundamentals (AI-901) Practical, real-world project work designed to help you develop hands-on skills in Excel, SQL, Python and PowerBI Dedicated 1-to-1 tutor support throughout your learning journey CV support, interview preparation, and career coaching Access to our employer network and recruitment partners Starting Salaries Upon successful completion of the programme, we guarantee a starting salary of up to £40,000. Who Is This Programme For? This programme is designed for individuals with little or no previous experience. You may be: Looking for a career change A recent school, college or university leaver Currently working in another industry Leaving the Armed Forces Interested in technology and problem-solving The most important qualities are a willingness to learn, good communication skills and a genuine interest in changing your career. Please note: this is a self-funded programme costing around £190 per month Our Job Guarantee Successfully complete the programme and meet the agreed career support requirements, and we guarantee you'll receive a job offer within 6 months. If not, we'll refund 100% of your course fees. Full T&Cs available. Ready to Start Your New Career? Click 'Apply Now' to speak with an Advisor and take the first step towards a rewarding career in one of the UK's fastest-growing sectors.
