Responsibilities
- Design, build, and maintain ETL/ELT pipelines
- Ingest data from internal systems and external APIs
- Develop scalable data processing workflows (batch and/or real-time)
- Ensure pipelines are reusable, efficient, and maintainable
- Integrate data from various sources, including transactional flows
- Handle complex data scenarios such as: Event sequencing (e.g. bet -> resolve), Idempotency and duplicate handling, Partial or delayed data
- Ensure consistency between source systems and analytical datasets
- Implement transformation logic aligned with architectural data models
- Build and maintain structured data layers for analytics consumption
- Collaborate closely with the Data Architect on model implementation
- Implement data validation, monitoring, and alerting mechanisms
- Identify and resolve data inconsistencies or failures
- Ensure high levels of data accuracy and availability
- Optimise pipelines for performance and cost-efficiency
- Support scaling of data infrastructure as volumes grow
- Ensure low-latency data availability where required
- Work closely with Data Architect, Analysts, and Manager
- Support Analysts by ensuring availability of curated datasets
- Contribute to continuous improvement of data platform capabilities
Requirements
- 3-7+ years experience in data engineering, backend engineering, or similar roles
- Strong programming skills (e.g. Python, SQL)
- Solid understanding of ETL/ELT processes and data pipeline design
- Experience working with APIs and integrating distributed systems
- Experience handling transactional or event-based data
- Strong understanding of: Data transformation techniques, Data warehousing concepts, Data modelling fundamentals
- Experience with data orchestration and workflow tools
- Ability to build robust, fault-tolerant systems
- Strong problem-solving skills with attention to detail and data accuracy.
Core Competencies
Demonstrates expertise in designing and maintaining ETL/ELT pipelines, ensuring data accuracy and availability while optimizing performance and cost-efficiency. Proficient in data transformation techniques and integrating various data sources to support analytics.
Tools & Technologies
- APIs
- Data Processing Workflows
- Data Infrastructure