Responsibilities
- Build and optimise the platform used by the data team to transform raw data into business insights.
- Build new data sources and pipelines that deliver key data and insights to the business, including integration with third party systems and ingestion through APIs.
- Scope and deliver new data engineering projects in collaboration with business stakeholders.
- Develop and deploy ML infrastructure to build out our ever growing AI requirements and use cases.
- Collaborate closely with our dev teams to build seamless integrations between our back end databases and our data platform.
- Shape the direction of our growing team and coach team members on best practices.
Qualifications
- First and foremost, a passion for decarbonisation.
- A passion for writing high quality code and building lean processes.
- Experience with distributed data processing.
- Experience with monitoring, testing, and data quality.
- Ability to work with ambiguity and own problems end to end.
- Experience building robust pipelines from diverse sources such as SQL & No SQL databases and API endpoints.
- Experience or expertise with:
- Airflow
- AWS
- Kubernetes
- Spark & distributed computing
- dbt
- Terraform
- Databricks (desirable but not required)
- MLOps implementation (desirable but not required)
- Distributed system optimisation techniques.
- Governance.
Equal Opportunity Statement
As an equal opportunity employer, we do not discriminate on the basis of any protected attribute. Our commitment is to provide equal opportunities, an inclusive work environment, and fairness for everyone.