Staff Data Engineer

  • Deepstreamtech
  • 13/05/2026
Full time Information Technology Telecommunications Python Data Scientist Testing

Job Description

Requirements
  • Proven experience operating at staff level (ownership of systems, not just pipelines)
  • Experience building and scaling modern data platforms
  • A track record of operating at staff or principal level: you've owned systems, shaped technical direction across teams, and influenced how engineering gets done - not just delivered pipelines
  • Deep experience building and scaling production data platforms, including high-ingestion time series workloads, and strong hands on ability in Python and modern data stack components (orchestration, warehousing, observability)
  • The ability to design for reliability and scale - you understand the trade offs in data system design and have made consequential architecture decisions you can speak to clearly
  • A product mindset: you care about whether the data is actually useful and used, not just whether the pipeline ran green
  • Experience with cloud data infrastructure (AWS or GCP) and a point of view on what good looks like
  • The communication skills to lead without authority - influencing technical direction across teams and making the case for the right thing even when it's harder
  • Strong programming skills in Python, with experience building production grade data systems
  • Experience with modern data stack components (e.g.):
  • Orchestration: Airflow / Dagster
  • Warehousing: Snowflake / BigQuery / Redshift / ClickHouse
  • Streaming (nice to have): Kafka / Flink
  • Experience with cloud platforms (AWS / GCP)
  • Experience with data observability and testing practices
  • (Desirable) Experience in energy or climate tech
  • (Desirable) Familiarity with time series data at scale
  • (Desirable) Experience supporting ML pipelines in production
  • (Desirable) Background in high growth startups or scale ups
What the job involves
  • We're looking for a Staff Data Engineer to lead the design and evolution of our data platform. This is a high impact, hands on role combining technical leadership, system architecture, and product thinking
  • You'll work closely with engineering, data science, and energy domain experts to ensure that data is reliable, scalable, and directly drives business value
  • You'll work across the data management service team alongside data and analytics engineers, and in close partnership with energy domain experts, data scientists, and the broader engineering organisation. This is a hands on senior technical leadership role - you'll be reviewing pull requests and setting architectural direction in the same week
  • What makes this genuinely different: you're not inheriting someone else's vision of what a data platform should be. The cultural norms, the standards, the practices - these are yours to define. If you've wanted to build the right thing from the ground up, this is that opportunity
  • Technical Leadership
  • Shape the technical direction across batch and streaming pipelines, setting the architecture others build to
  • Set standards for pipeline design and data quality
  • Lead design reviews and mentor other data engineers
  • Evaluate and introduce tooling where it raises the bar - and make the case for when it doesn't
  • Data Platform & Pipelines
  • Build and maintain robust ETL/ELT pipelines
  • Build systems optimised for high ingestion, low latency querying of time series data (TSDS)
  • Optimise pipelines for performance, cost, and reliability
  • Enable self serve analytics and decision making across the company
  • Reliability and observability
  • Implement data quality frameworks with real teeth: SLAs, automated testing, lineage, and monitoring
  • Establish practices specific to energy data: late arrivals, reprocessing, backfills, and the failure modes that matter in this domain
  • Build the observability layer that makes the platform trustworthy without constant human oversight
  • Scale and performance
  • Identify and fix the bottlenecks that constrain us today
  • Optimise pipelines for performance, cost, and reliability as data volumes grow
  • Architect for the next order of magnitude, not just the next quarter
  • Technical leadership and culture
  • Set engineering standards for pipeline design, data quality, and system observability
  • Lead design reviews and mentor data engineers, raising the bar for how the team works
  • Act as a multiplier: the people around you should get better because of how you approach problems
  • What Success Looks Like:
What Success Looks Like
  • The data platform handles tem's current scale without firefighting, and is architected for the next phase of growth
  • Other teams can access, trust, and use data without routing requests through the data engineering team
  • There is a tight, reliable feedback loop between data ingestion and consumption: trading, forecasting, and analytics teams make faster decisions because the data is there when they need it
  • The data engineering team has clearer standards, better practices, and higher output than when you arrived