Principal Data Engineer

We're 1st Central, a market leading insurance company utilising smart data and technology at pace.

We are recruiting a Principal Data Engineer to serve as a key bridge between business opportunities and technical solutions.

• Provide technical leadership on data solutions across both project delivery and ongoing data products, defining the technical direction of the data platform and solution design.
• Collaborate closely with architects to ensure alignment with overall platform patterns and standards.
• Set and enforce coding, low level design, and ingestion standards for the data platform.
• Design and build high complexity, secure, governed, and high performance data pipelines from on premise, off premise, internal, and external data sources.
• Define data quality, observability, and self testing patterns to proactively identify processing issues.
• Transform data into lake storage, database models, reporting structures, and analytics applications.
• Develop physical data models that optimise storage and meet business needs.
• Perform unit testing of own code and peer testing of others' code to ensure quality and integrity.
• Document data platform components to provide transparency and understandability.
• Coach and mentor senior and junior data engineers.
• Create high complexity BI solutions, including data marts, semantic layers, and visualisation solutions in tools such as Power BI.

• 10+ years of experience designing and building end to end data solutions.
• Extensive experience with Microsoft Azure data platforms, including Azure Data Lake Storage Gen2, Azure Synapse Analytics, and Databricks.
• Strong knowledge of Delta Lake lakehouse architectures, Medallion patterns, and scalable batch and streaming workloads.
• Proficiency in Python, SQL, and experience with PySpark, Spark SQL, and other data engineering languages.
• Hands on experience with Git based workflows, CI/CD pipelines (YAML), and infrastructure as code tools such as Terraform or Bicep.
• Experience designing data models (Kimball dimensional modelling) and implementing data quality, cleansing, and validation logic.
• Deep understanding of Azure security, network isolation, identity and access management, and cost management.
• Excellent coaching, mentoring, and leadership skills, with a proven ability to guide engineering teams.
• Strong problem solving skills and the ability to simplify complex problems.
• Familiarity with Agile practices such as Scrum or Kanban.