DataBricks Specialist - Short Term Contract
This role is a DataBricks Specialist on a short-term contract with an immediate start. Key skills include Databricks, Python, SQL, and DataOps practices. Experience with ETL/ELT frameworks and cloud platforms (AWS, Azure, GCP) is required.
Location: United Kingdom (London, England, United Kingdom)
Start: Immediate
Responsibilities
- Lead design and development of robust data pipelines.
- Integrate data from diverse sources (APIs, relational databases, files, etc.).
- Collaborate with business and analytics teams to understand data requirements.
- Ensure quality, reliability, security and governance of the ingested data.
- Follow modern DataOps practices such as code versioning, data tests, and CI/CD.
- Document processes and best practices in data engineering.
Required Skills and Qualifications
- Proven experience in building and managing large-scale data pipelines in Databricks (PySpark, Delta Lake, SQL).
- Strong programming skills in Python and SQL for data processing and transformation.
- Deep understanding of ETL/ELT frameworks, data warehousing, and distributed data processing.
- Hands-on experience with modern DataOps practices: version control (Git), CI/CD pipelines, automated testing, infrastructure-as-code.
- Familiarity with cloud platforms (AWS, Azure, or GCP) and related data services.
- Strong problem-solving skills with the ability to troubleshoot performance, scalability, and reliability issues.
- Proficiency in Git.