Senior Data Engineer About the role
As a Senior Data Engineer at Informed Solutions, you'll take a leading role in designing, building, and delivering robust data platforms and pipelines that underpin critical public sector services. Your work will enable better decision making, improve service outcomes, and help create a healthier, safer, and more sustainable UK.
Role Responsibilities
- Own delivery of data services: Take responsibility for the engineering and delivery of small-to-medium data services (or major components of larger platforms) across the full lifecycle, from design and build through to deployment, operation, and continual improvement.
- Lead on data architecture and design: Take ownership of the end-to-end design of data pipelines, data models, and platform components, producing and maintaining high quality technical documentation. Collaborate with Lead and Principal Engineers, Data Architects, and practitioners from disciplines such as UCD and Data Science to review and refine designs and translate them into robust implementations.
- Build and operate data pipelines: design, develop, test, and deploy ETL pipelines to source, cleanse, and optimise data at scale.
- Automate and orchestrate data workflows: Develop and maintain orchestration solutions (e.g. Apache Airflow), implementing testing, alerting, monitoring, and data quality checks as first class concerns.
- Ensure data quality and observability: Embed data testing and validation processes alongside monitoring and alerting systems such as Prometheus and Grafana to ensure reliability, performance, and operational insight across data services.
- Work with metadata and standards: Implement metadata capture and validation within data pipelines, including working with geospatial metadata standards where relevant, to support governance, discovery, and reuse.
- Support delivery planning: Partner with Delivery Managers to plan and structure technical activities, provide accurate and commercially sound estimates, and proactively identify and mitigate technical risks, issues, and dependencies.
- Build stakeholder trust: Act as a trusted technical advisor to client counterparts, partner organisations, and third parties, ensuring alignment between data requirements, business needs, and technical delivery.
- Mentor and develop others: Provide guidance and technical leadership to Data Engineers and adjacent roles, supporting professional growth and fostering a culture of high quality, sustainable delivery.
Essential Skills
- 5+ years' proven experience in a Data Engineering role, delivering production grade data platforms and pipelines.
- Strong Python skills, with experience using Python to build, test, and operate data pipelines and services.
- Proficiency with at least one major cloud platform (AWS, Azure, or GCP), delivering scalable, secure, and well architected data solutions.
- Strong experience working with core data stores, including relational databases such as PostgreSQL and Microsoft SQL.
- Experience with non relational and NoSQL data stores (e.g. DynamoDB, MongoDB or equivalent) and understanding when to apply them.
- Hands on experience designing and operating event streaming platforms including Kafka and its core concepts on topics, messages, and consumers.
- Experience building and operating automated data workflows using tools like Apache Airflow.
- Experience implementing data quality and validation checks throughout pipelines to ensure accuracy, completeness, consistency, and reliability.
- Practical experience implementing monitoring, observability and alerting for data platforms using tools such as Prometheus and Grafana.
- Strong understanding of data architecture patterns, including data lakes, data warehouses, and event driven architectures.
- Awareness of GDPR and data protection practices, including secure handling of sensitive data and privacy by design principles.
- Experience working in Agile environments (Scrum, Kanban), contributing to delivery planning, estimation, and iterative improvement.
- Proven experience mentoring and supporting other engineers, enabling high quality delivery and team capability growth.
- Familiarity with metadata standards, including experience validating and implementing metadata, with exposure to geospatial metadata standards (e.g. ISO 19115, INSPIRE, GEMINI) being highly desirable.
Desirable Skills
- Experience working in a consultancy environment, engaging with external stakeholders and clients.
- Exposure to public sector projects and delivering solutions in complex organisational landscapes.
- Experience working with geospatial or GIS data (e.g. spatial databases, geospatial data formats or services) is highly desirable, reflecting the nature of the work we deliver.
- Experience with Graph databases (e.g. Neo4j, Apache Jena) for modelling and querying complex relationships in data.
Key abilities that are necessary for success
- Able to work effectively in environments where you can adapt to innovative practices involving uncertainty, change, and agile working.
- Ability to manage competing priorities and your time so you can deliver high quality work within deadlines.
- Good interpersonal skills, with the ability to respect and accept different perspectives and disagreement, work collaboratively in diverse teams, and maintain positive working relationships.
- Good judgement and decision making in complex environments where uncertainty, ambiguity, and risk must be collaboratively managed by working supportively alongside others.
- Good communication skills, including clear, constructive written outputs and measured, respectful verbal delivery.
What You'll Gain
At Informed Solutions, you'll have the opportunity to make a tangible impact while advancing your career. You'll gain:
- Ownership and influence: Lead the delivery of services, shape technical direction, and influence project outcomes at a senior level.
- Professional growth: Develop your leadership and mentoring skills, guiding colleagues and helping them thrive.
- Exposure to high profile projects: Work on enterprise scale digital solutions that improve public services and have societal impact.
- Technical development: Expand your expertise across a wide range of modern technologies, including cloud, microservices and serverless architectures.
- Collaborative, supportive environment: Join a team that values learning, knowledge sharing, and continuous improvement.
- Career progression: Opportunities to grow within a consultancy setting, broadening your client facing experience and technical leadership.
Benefits package includes
- InformedACADEMY - We offer excellent career development opportunities through our award winning personal and professional development programmes, including support with professional certifications.
- Industry leading health and wellbeing plan - We partner with several wellbeing support functions to cater to each individual's need, including 24/7 GP services, mental health support and physical health support.
- Hybrid working
- Private Health Care Cover
- Generous life assurance cover
- Gym Membership
- Monthly office lunch
- Onsite massage sessions
- 25 paid working days holiday per year plus bank holidays
- Sabbatical Leave Scheme
- Enhanced Maternity Leave and Pay
- Enhanced Paternity Leave and Pay
- Company Pension Contribution
- Profit Share Scheme
- Payment of professional subscriptions
- Generous referral scheme with no limits on the number of referrals
Qualifying period applies