Senior Data Engineer - £63,500 - Hybrid

£60,000 to £63,500 per year, £63,500 (DOE)

Contract Type: Permanent

Hours: Full time

Disability Confident: No

Closing Date: 20/07/2026

About this job

• Hybrid Working (2 days a week in office)

The company is a not for profit housing association established in 2003 as the first large scale voluntary transfer of council owned homes to a social housing provider in Wales.

The organisation is rooted in its mission to provide safe, high quality and affordable homes across Bridgend and South Wales, creating thriving communities where people feel secure, supported, and proud to live.

Your new role

The overall purpose of this role is to be the primary architect and builder of our modern data estate.

This is a high impact, "greenfield" role where you will design and implement the Azure Databricks infrastructure that forms the backbone of our future data, BI, and AI capabilities, and continue to mature our PowerBI architecture for ease and robustness of reporting and self service.

This role will champion a DataSecOps approach, ensuring that security, privacy and compliance are embedded throughout the data lifecycle from the outset. The nascent nature of our data capabilities means you will have the opportunity to help shape those capabilities to achieve lasting positive value for the organisation as a whole, our customers and our colleagues.

No Sponsorship Offered for this role.

Data Platform Architecture & DataSecOps

• Lead the design and implementation of a scalable and secure data platform using Azure Databricks, enabling the ingestion, transformation and governance of data from key business systems while supporting a reliable "single version of the truth" for reporting and analytics;
• Design and deploy secure, scalable ELT or ETL pipelines using Databricks (Unity Catalog), Delta Lake, and Azure Data Lake Storage (ADLS);
• Implement shift left security principles within the data ingestion layer, including automated data masking, encryption at rest and in transit, and row and column level security controls;
• Establish and embed DataOps and DataSecOps practices, including CI/CD, version control (Git), and automated security scanning of data pipelines.

Power BI & Reporting Foundation

• Finalise a governed Power BI architecture, including standardised semantic models and dataflows;
• Ensure robust Identity and Access Management (IAM) across Power BI workspaces, enabling true self service analytics without compromising data privacy.
• Proactively identify data and logic inconsistencies between systems, providing clear evidence and insight to support stakeholders in resolving operational issues;
• Act as a technical lead, coaching a Junior Data Engineer and Analyst on modern engineering standards and secure coding practices;
• Work with the Software Engineering Manager to establish naming conventions, metadata standards, and data residency/sovereignty protocols.

Stakeholder Engagement & Continuous Improvement

• Constructively challenge stakeholders and teams where necessary on scope and delivery to ensure deliverables remain on track;
• Support continuous improvement efforts through ongoing analysis and feedback, and implementation of best practices;
• Coach and mentor colleagues to improve the maturity of the capability;
• Work with housing and frontline teams to ensure system implementations improve service quality and accessibility for customers;
• Identify opportunities to leverage technology to enhance customer engagement, service delivery, and operational efficiencies.

What you'll need to succeed (Knowledge/Qualifications)

Expert level SQL and Python (ideally PySpark).

A strong understanding of DataSecOps; experience with PII handling, data obfuscation, and RBAC (Role Based Access Control).

Proven ability to design data systems (preferably Medallion architectures).

Advanced knowledge of Power BI security and administration.

Familiarity with git and gitflow version control processes;

Knowledge of cloud cost and resource optimisation (FinOps principles)

Awareness of agile delivery methodologies (Scrum or Kanban).

Desirable

Experience with DBT (Data Build Tool) or Prefect.

Awareness of AI/ML data preparation and the security implications of LLM training data.

Knowledge of housing, asset management, or repairs systems (ideally Civica Cx and Total Mobile Service Connect).

Experience

Implemented and operated cloud based data platforms, ideally on Azure (experience MUST include build, not just maintenance of existing).

Delivered a Delta Lakehouse, Medallion Architecture, or similar curated data layering approach.

Embedded security controls into pipelines (e.g., secure credential handling, encryption, RBAC, ACLs).

Worked with CI/CD, version control (Git), code review processes, and automated testing for data.

Designed data models (star schema, transformed delta tables, data marts) for analytics and BI.

Ensured data quality through validation, profiling, reconciliation, and automated tests.

Improved or restructured existing reporting environments to make them scalable, governed, and self service friendly.

Provided guidance, reviewed code, coached junior team members, and influenced team practices.

Design of data architecture, as well as build.

Experience managing multiple workstreams simultaneously, balancing major projects with smaller initiatives.