Data Platform Engineer London (Hybrid) 6 Month Contract £(Apply online only)/day (Inside IR35) Data Platform Engineer needed with strong AWS cloud platform migration and Terraform expertise. 6 Month Rolling Contract based in London (Hybrid), with occasional travel to the Leeds office. Hybrid Working - 3-4 days/week remote (WFH) + 1-2 days/week working from the office in London . A chance to work with a leading global IT transformation business specialising in large-scale Government projects: In-depth Data Engineering experience + AWS cloud data platform migration experience. The role will focus on Data Engineering, cloud platform migration, infrastructure automation + supporting the delivery of large-scale data processing solutions. Supporting a major Data platform modernisation programme including migration of data pipelines, notebooks + data workloads from legacy Databricks environment to modern AWS cloud-based data platforms. Strong Terraform + Infrastructure as Code (IaC) skills including developing + maintaining Terraform code to support data platform deployments. Supporting deployment of data platform components, jobs, permissions + environment configurations. Analysing current Data Engineering solutions + identifying data migration requirements. Onboarding datasets, users + workloads onto target platform. Data Engineering: Python, PySpark, SQL, Data Pipeline Development, ETL/ELT Design, Data Modelling, Data Quality, Data Validation. AWS Cloud Platform, S3, IAM, CI/CD concepts, DevOps principles, Agile Delivery. Advantageous Skills: Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE) experience, NHS / Healthcare data experience. Government / (url removed) / Public Sector domain experience strongly preferred. Must be active SC Cleared or fully eligible to undergo the full SC Security Clearance process.
16/06/2026
Contractor
Data Platform Engineer London (Hybrid) 6 Month Contract £(Apply online only)/day (Inside IR35) Data Platform Engineer needed with strong AWS cloud platform migration and Terraform expertise. 6 Month Rolling Contract based in London (Hybrid), with occasional travel to the Leeds office. Hybrid Working - 3-4 days/week remote (WFH) + 1-2 days/week working from the office in London . A chance to work with a leading global IT transformation business specialising in large-scale Government projects: In-depth Data Engineering experience + AWS cloud data platform migration experience. The role will focus on Data Engineering, cloud platform migration, infrastructure automation + supporting the delivery of large-scale data processing solutions. Supporting a major Data platform modernisation programme including migration of data pipelines, notebooks + data workloads from legacy Databricks environment to modern AWS cloud-based data platforms. Strong Terraform + Infrastructure as Code (IaC) skills including developing + maintaining Terraform code to support data platform deployments. Supporting deployment of data platform components, jobs, permissions + environment configurations. Analysing current Data Engineering solutions + identifying data migration requirements. Onboarding datasets, users + workloads onto target platform. Data Engineering: Python, PySpark, SQL, Data Pipeline Development, ETL/ELT Design, Data Modelling, Data Quality, Data Validation. AWS Cloud Platform, S3, IAM, CI/CD concepts, DevOps principles, Agile Delivery. Advantageous Skills: Databricks, Palantir Foundry, Foundry Pipeline Builder, Ontology concepts, Delta Lake, Secure Data Environment (SDE) experience, NHS / Healthcare data experience. Government / (url removed) / Public Sector domain experience strongly preferred. Must be active SC Cleared or fully eligible to undergo the full SC Security Clearance process.
Data Platform Manager UK / Remote Circa 65k - 75k per annum + bonus & benefits Are you ready to take ownership of a data platform and shape how data drives business decisions? This is a rare opportunity to step into a leadership role where you'll combine hands-on data engineering with strategic product management, all while building and mentoring a high-performing team. You'll play a pivotal role in evolving a modern data platform (Microsoft Fabric / Azure Synapse), ensuring it delivers real business value, scalability, and innovation. If you're passionate about data, leadership, and building robust data solutions-this is one you don't want to miss. What you'll be doing Own the end-to-end lifecycle of a core data platform, from design and build through to release, optimisation, and retirement Collaborate closely with product owners and data scientists to align platform capabilities with business needs Lead datalake migration initiatives, including consolidation of legacy data, medallion architecture design, schema evolution, and cutover planning Provide hands-on data engineering expertise, building and optimising PySpark pipelines Ensure data is secure, high-quality, accessible, and structured for maximum business value Drive best practice in data governance, cataloguing, lineage, and standards across the organisation Promote data re-use and reduce duplication across systems Oversee ETL processes, data integration, and BI architecture Manage and develop a small, high-performing data team, including mentoring, 1:1s, and continuous improvement initiatives Work closely with stakeholders across operations, analytics, and reporting teams Ensure compliance with data protection regulations and internal policies What you'll bring Strong hands-on experience in data engineering, including PySpark and production data pipelines Experience with Microsoft Fabric, Azure Synapse, or similar platforms (e.g. Databricks, AWS) Proven ability to design and manage scalable data pipelines, models, and schemas Solid SQL and relational database expertise Experience with datalake migration concepts (medallion architecture, incremental loads, backfill, cutover) Strong understanding of data governance, quality, and lifecycle management Experience with BI tools such as Power BI or Tableau Knowledge of DevOps practices (Git, Azure DevOps, CI/CD pipelines) Strong Python skills and experience with APIs/data integration tools Understanding of data protection regulations Leadership experience with the ability to mentor and develop teams Excellent communication, problem-solving, and stakeholder management skills Desirable: Experience with project management methodologies, SDLC tools, and awareness of emerging data trends What's in it for you? Performance-related bonus Private healthcare and life assurance 33 days annual leave (rising to 38 with service) Discounts across major retailers, restaurants, and holidays Generous family benefits and staff discounts Award-winning training and clear career progression Comprehensive wellbeing support for you and your family If you've held any of these roles or used these technologies/skills, this role could be a great fit: Data Product Manager, Data Engineering Lead, Lead Data Engineer, Azure Data Engineer, Data Platform Manager, Analytics Engineering Manager, Cloud Data Engineer, BI & Data Lead, PySpark Engineer, Databricks Engineer, Head of Data Engineering. Deerfoot Recruitment Solutions Ltd is a leading independent tech recruitment consultancy in the UK. For every CV sent to clients, we donate 1 to The Born Free Foundation. We are a Climate Action Workforce in partnership with Ecologi. If this role isn't right for you, explore our referral reward program with payouts at interview and placement milestones. Visit our website for details. Deerfoot Recruitment Solutions Ltd is acting as an Employment Agency in relation to this vacancy.
16/06/2026
Full time
Data Platform Manager UK / Remote Circa 65k - 75k per annum + bonus & benefits Are you ready to take ownership of a data platform and shape how data drives business decisions? This is a rare opportunity to step into a leadership role where you'll combine hands-on data engineering with strategic product management, all while building and mentoring a high-performing team. You'll play a pivotal role in evolving a modern data platform (Microsoft Fabric / Azure Synapse), ensuring it delivers real business value, scalability, and innovation. If you're passionate about data, leadership, and building robust data solutions-this is one you don't want to miss. What you'll be doing Own the end-to-end lifecycle of a core data platform, from design and build through to release, optimisation, and retirement Collaborate closely with product owners and data scientists to align platform capabilities with business needs Lead datalake migration initiatives, including consolidation of legacy data, medallion architecture design, schema evolution, and cutover planning Provide hands-on data engineering expertise, building and optimising PySpark pipelines Ensure data is secure, high-quality, accessible, and structured for maximum business value Drive best practice in data governance, cataloguing, lineage, and standards across the organisation Promote data re-use and reduce duplication across systems Oversee ETL processes, data integration, and BI architecture Manage and develop a small, high-performing data team, including mentoring, 1:1s, and continuous improvement initiatives Work closely with stakeholders across operations, analytics, and reporting teams Ensure compliance with data protection regulations and internal policies What you'll bring Strong hands-on experience in data engineering, including PySpark and production data pipelines Experience with Microsoft Fabric, Azure Synapse, or similar platforms (e.g. Databricks, AWS) Proven ability to design and manage scalable data pipelines, models, and schemas Solid SQL and relational database expertise Experience with datalake migration concepts (medallion architecture, incremental loads, backfill, cutover) Strong understanding of data governance, quality, and lifecycle management Experience with BI tools such as Power BI or Tableau Knowledge of DevOps practices (Git, Azure DevOps, CI/CD pipelines) Strong Python skills and experience with APIs/data integration tools Understanding of data protection regulations Leadership experience with the ability to mentor and develop teams Excellent communication, problem-solving, and stakeholder management skills Desirable: Experience with project management methodologies, SDLC tools, and awareness of emerging data trends What's in it for you? Performance-related bonus Private healthcare and life assurance 33 days annual leave (rising to 38 with service) Discounts across major retailers, restaurants, and holidays Generous family benefits and staff discounts Award-winning training and clear career progression Comprehensive wellbeing support for you and your family If you've held any of these roles or used these technologies/skills, this role could be a great fit: Data Product Manager, Data Engineering Lead, Lead Data Engineer, Azure Data Engineer, Data Platform Manager, Analytics Engineering Manager, Cloud Data Engineer, BI & Data Lead, PySpark Engineer, Databricks Engineer, Head of Data Engineering. Deerfoot Recruitment Solutions Ltd is a leading independent tech recruitment consultancy in the UK. For every CV sent to clients, we donate 1 to The Born Free Foundation. We are a Climate Action Workforce in partnership with Ecologi. If this role isn't right for you, explore our referral reward program with payouts at interview and placement milestones. Visit our website for details. Deerfoot Recruitment Solutions Ltd is acting as an Employment Agency in relation to this vacancy.
About the Role We're seeking a Senior Data Engineer to transform our analytics infrastructure from monolithic SQL/PostgreSQL systems into a modern Azure-based data platform with DevOps at its core. This permanent position focuses on building an automated, self-service architecture designed for continuous evolution. You'll bring 5+ years of data engineering experience with proven expertise in implementing enterprise-scale data platforms and DevOps methodologies. You'll architect solutions using modern patterns, establish automated data pipelines with CI/CD practices, implement infrastructure-as-code, and mentor our team while building systems that adapt to emerging technologies. We need someone who prioritises maintainable, scalable, and adaptable systems. We're committed to Azure but flexible on specific products - we value pragmatic solutions based on actual needs. Structure & Expectations Reports to: Product organisation - you'll work closely with product teams to align data capabilities with business objectives Location: London office 3 days per week, flexible remote work the rest of the time Team: You'll be the senior technical lead and engineer for data platform initiatives, working with business analysts, engineers, and product stakeholders Timeline: Initial focus on migration (6 months), then platform evolution, DevOps maturity, and capability expansion Core Technical Requirements Azure Data Platform Architecture Microsoft Azure data stack expertise: ADF, Delta Lake, ADLS Gen2, Synapse, Databricks (with flexibility to integrate non-Microsoft solutions) Experience with PostgreSQL and SQL Server, including data replication/integration patterns for migration Dimensional and relational data modeling expertise Medallion architecture implementation (bronze silver gold) Data Engineering & Pipelines Proficiency in Python and SQL for ELT/ETL pipelines Batch, streaming, and micro-batching architectures CDC patterns and incremental load strategies at scale Experience with Delta Change Data Feed or equivalent real-time sync mechanisms Experience building ingestion pipelines from diverse external APIs (CRM, analytics, finance, observability tools) into a unified lakehouse DevOps & Infrastructure Infrastructure as Code (Terraform preferred) CI/CD with Azure DevOps or GitHub Actions Test automation for data pipelines (unit, integration, regression) Azure networking and security best practices Container orchestration (Kubernetes/AKS) experience beneficial Event-driven architectures and API integration patterns Monitoring and alerting with Azure Monitor and Log Analytics Performance tuning and cost optimisation Git version control proficiency Governance, Compliance & Security Strong understanding of information security requirements Experience with Microsoft Purview or similar governance frameworks GDPR/data protection implementation, including the right-to-be-forgotten RBAC, encryption strategies, and audit logging Data lineage and quality enforcement Non-Technical Requirements Communication & Documentation Strong technical writing - please provide examples Clear presentation skills for technical and non-technical audiences Experience creating architecture diagrams, data flows, and runbooks Leadership Track record of training and upskilling teams Experience with structured project planning: from high-level initiatives through quarterly planning to executable tasks Comfort working with product managers and translating between technical and business contexts What You'll Do Architect a modern data platform with DevOps principles for long-term business evolution Establish automated engineering best practices for data, governance, and operations Build robust batch and real-time processing pipelines (integrating web application data with analytics sources and external APIs) Create comprehensive documentation and training materials Translate business requirements into maintainable technical solutions Mentor team members and drive continuous improvement Why This Matters You'll establish the foundation for our data platform's next decade. This platform will power our next generation of internal and customer products while empowering our business with real-time insights for faster, smarter decisions.
16/06/2026
Full time
About the Role We're seeking a Senior Data Engineer to transform our analytics infrastructure from monolithic SQL/PostgreSQL systems into a modern Azure-based data platform with DevOps at its core. This permanent position focuses on building an automated, self-service architecture designed for continuous evolution. You'll bring 5+ years of data engineering experience with proven expertise in implementing enterprise-scale data platforms and DevOps methodologies. You'll architect solutions using modern patterns, establish automated data pipelines with CI/CD practices, implement infrastructure-as-code, and mentor our team while building systems that adapt to emerging technologies. We need someone who prioritises maintainable, scalable, and adaptable systems. We're committed to Azure but flexible on specific products - we value pragmatic solutions based on actual needs. Structure & Expectations Reports to: Product organisation - you'll work closely with product teams to align data capabilities with business objectives Location: London office 3 days per week, flexible remote work the rest of the time Team: You'll be the senior technical lead and engineer for data platform initiatives, working with business analysts, engineers, and product stakeholders Timeline: Initial focus on migration (6 months), then platform evolution, DevOps maturity, and capability expansion Core Technical Requirements Azure Data Platform Architecture Microsoft Azure data stack expertise: ADF, Delta Lake, ADLS Gen2, Synapse, Databricks (with flexibility to integrate non-Microsoft solutions) Experience with PostgreSQL and SQL Server, including data replication/integration patterns for migration Dimensional and relational data modeling expertise Medallion architecture implementation (bronze silver gold) Data Engineering & Pipelines Proficiency in Python and SQL for ELT/ETL pipelines Batch, streaming, and micro-batching architectures CDC patterns and incremental load strategies at scale Experience with Delta Change Data Feed or equivalent real-time sync mechanisms Experience building ingestion pipelines from diverse external APIs (CRM, analytics, finance, observability tools) into a unified lakehouse DevOps & Infrastructure Infrastructure as Code (Terraform preferred) CI/CD with Azure DevOps or GitHub Actions Test automation for data pipelines (unit, integration, regression) Azure networking and security best practices Container orchestration (Kubernetes/AKS) experience beneficial Event-driven architectures and API integration patterns Monitoring and alerting with Azure Monitor and Log Analytics Performance tuning and cost optimisation Git version control proficiency Governance, Compliance & Security Strong understanding of information security requirements Experience with Microsoft Purview or similar governance frameworks GDPR/data protection implementation, including the right-to-be-forgotten RBAC, encryption strategies, and audit logging Data lineage and quality enforcement Non-Technical Requirements Communication & Documentation Strong technical writing - please provide examples Clear presentation skills for technical and non-technical audiences Experience creating architecture diagrams, data flows, and runbooks Leadership Track record of training and upskilling teams Experience with structured project planning: from high-level initiatives through quarterly planning to executable tasks Comfort working with product managers and translating between technical and business contexts What You'll Do Architect a modern data platform with DevOps principles for long-term business evolution Establish automated engineering best practices for data, governance, and operations Build robust batch and real-time processing pipelines (integrating web application data with analytics sources and external APIs) Create comprehensive documentation and training materials Translate business requirements into maintainable technical solutions Mentor team members and drive continuous improvement Why This Matters You'll establish the foundation for our data platform's next decade. This platform will power our next generation of internal and customer products while empowering our business with real-time insights for faster, smarter decisions.
Senior Data Lead / Project Manager - ERP Transformation (SAP to Microsoft Dynamics) Location: Europe (Multi-Country) Remote with Travel Contract: 6+ Months We are supporting a major European transformation programme within a global FMCG environment, seeking a highly experienced Senior Data Lead / Project Manager to own and deliver the end-to-end data workstream across a complex, multi-country ERP migration. This is a critical role at the heart of a large-scale transformation, responsible for ensuring the successful migration of business-critical data from SAP ECC and associated systems into Microsoft Dynamics, with zero disruption to manufacturing and supply chain operations. The Role You will take full ownership of the data migration lifecycle - from strategy and scoping through to execution, validation, and cutover - working across multiple systems, countries, and business functions. This is a hands-on leadership role , requiring both deep technical expertise in SAP data structures and strong project delivery capability. Key Responsibilities Define and execute the end-to-end data migration strategy Lead data profiling, cleansing, transformation, and validation activities Manage multi-cycle migration rehearsals and final cutover Oversee data extraction and loading across SAP and non-SAP systems Ensure data quality, integrity, and reconciliation at every stage Drive compliance with GDPR, data governance, and regulatory standards Lead cross-functional teams including data analysts, ETL developers, and business stakeholders Provide regular reporting on data readiness, risks, and progress Key Requirements 8+ years' experience in data migration, ETL, and data management Strong hands-on expertise in SAP ECC data models (MM, SD, FI/CO, PP, etc.) Proven delivery of SAP to Microsoft Dynamics ERP migration (essential) Experience with Syniti ADMM / SAP Migration Cockpit / ETL tools Deep understanding of data quality, cleansing, and governance frameworks Experience in multi-country transformation programmes Strong stakeholder management and workshop facilitation skills Desirable Microsoft Dynamics F&O / Business Central knowledge SAP S/4HANA migration experience FMCG / manufacturing domain expertise Data governance and master data management (MDG) exposure Power BI or data quality dashboarding experience This is urgent opportunity where the hiring manager is looking to interview asap. If you are interested then apply with a copy of your CV or send your CV to khushboo .co .uk Randstad Technologies is acting as an Employment Business in relation to this vacancy.
14/06/2026
Contractor
Senior Data Lead / Project Manager - ERP Transformation (SAP to Microsoft Dynamics) Location: Europe (Multi-Country) Remote with Travel Contract: 6+ Months We are supporting a major European transformation programme within a global FMCG environment, seeking a highly experienced Senior Data Lead / Project Manager to own and deliver the end-to-end data workstream across a complex, multi-country ERP migration. This is a critical role at the heart of a large-scale transformation, responsible for ensuring the successful migration of business-critical data from SAP ECC and associated systems into Microsoft Dynamics, with zero disruption to manufacturing and supply chain operations. The Role You will take full ownership of the data migration lifecycle - from strategy and scoping through to execution, validation, and cutover - working across multiple systems, countries, and business functions. This is a hands-on leadership role , requiring both deep technical expertise in SAP data structures and strong project delivery capability. Key Responsibilities Define and execute the end-to-end data migration strategy Lead data profiling, cleansing, transformation, and validation activities Manage multi-cycle migration rehearsals and final cutover Oversee data extraction and loading across SAP and non-SAP systems Ensure data quality, integrity, and reconciliation at every stage Drive compliance with GDPR, data governance, and regulatory standards Lead cross-functional teams including data analysts, ETL developers, and business stakeholders Provide regular reporting on data readiness, risks, and progress Key Requirements 8+ years' experience in data migration, ETL, and data management Strong hands-on expertise in SAP ECC data models (MM, SD, FI/CO, PP, etc.) Proven delivery of SAP to Microsoft Dynamics ERP migration (essential) Experience with Syniti ADMM / SAP Migration Cockpit / ETL tools Deep understanding of data quality, cleansing, and governance frameworks Experience in multi-country transformation programmes Strong stakeholder management and workshop facilitation skills Desirable Microsoft Dynamics F&O / Business Central knowledge SAP S/4HANA migration experience FMCG / manufacturing domain expertise Data governance and master data management (MDG) exposure Power BI or data quality dashboarding experience This is urgent opportunity where the hiring manager is looking to interview asap. If you are interested then apply with a copy of your CV or send your CV to khushboo .co .uk Randstad Technologies is acting as an Employment Business in relation to this vacancy.
Data Migration Lead Core Banking Transformation Contract: 6 Months IR35: Inside IR35 Location: Remote (UK Based) Rates: Open We are currently recruiting for an experienced Data Migration Lead to support a major Core Banking transformation programme for a leading banking client. This is a 6-month contract opportunity for a specialist who can take ownership of the end-to-end data migration strategy through to execution . The successful candidate will play a key role in defining the migration approach, managing delivery, ensuring data quality, and supporting successful migration outcomes. We are looking for someone with strong experience delivering complex data migration programmes within Banking or Financial Services environments . Experience of specific core banking platforms is not essential; we are looking for proven Core Banking Data Migration expertise and the ability to lead large-scale migration activity. Key Responsibilities Own and deliver the end-to-end data migration strategy and execution plan . Lead data discovery, profiling, mapping, cleansing, transformation, validation, and reconciliation activities. Define migration approach, governance, controls, risks, and delivery milestones. Work closely with business stakeholders, technical teams, architects, and SMEs. Support migration testing, mock migrations, cutover planning, and execution. Ensure strong data quality and accuracy throughout the migration lifecycle. Required Experience Proven experience as a Data Migration Lead / Data Migration Manager . Strong background delivering banking or financial services data migrations . Experience managing complex core banking system migrations . Strong understanding of data migration methodologies, processes, and governance. Excellent stakeholder management and communication skills. Ability to lead migration activity across business and technology teams. Desirable Experience Experience delivering large-scale banking transformation programmes. Knowledge of ETL processes, data migration tooling, and data governance. Experience working within regulated financial services environments. This is an excellent opportunity for an experienced Data Migration Lead to join a significant banking transformation programme. Remote candidates will be considered.
12/06/2026
Contractor
Data Migration Lead Core Banking Transformation Contract: 6 Months IR35: Inside IR35 Location: Remote (UK Based) Rates: Open We are currently recruiting for an experienced Data Migration Lead to support a major Core Banking transformation programme for a leading banking client. This is a 6-month contract opportunity for a specialist who can take ownership of the end-to-end data migration strategy through to execution . The successful candidate will play a key role in defining the migration approach, managing delivery, ensuring data quality, and supporting successful migration outcomes. We are looking for someone with strong experience delivering complex data migration programmes within Banking or Financial Services environments . Experience of specific core banking platforms is not essential; we are looking for proven Core Banking Data Migration expertise and the ability to lead large-scale migration activity. Key Responsibilities Own and deliver the end-to-end data migration strategy and execution plan . Lead data discovery, profiling, mapping, cleansing, transformation, validation, and reconciliation activities. Define migration approach, governance, controls, risks, and delivery milestones. Work closely with business stakeholders, technical teams, architects, and SMEs. Support migration testing, mock migrations, cutover planning, and execution. Ensure strong data quality and accuracy throughout the migration lifecycle. Required Experience Proven experience as a Data Migration Lead / Data Migration Manager . Strong background delivering banking or financial services data migrations . Experience managing complex core banking system migrations . Strong understanding of data migration methodologies, processes, and governance. Excellent stakeholder management and communication skills. Ability to lead migration activity across business and technology teams. Desirable Experience Experience delivering large-scale banking transformation programmes. Knowledge of ETL processes, data migration tooling, and data governance. Experience working within regulated financial services environments. This is an excellent opportunity for an experienced Data Migration Lead to join a significant banking transformation programme. Remote candidates will be considered.
Database & Data Migration Engineer (Access to SQL Server) Inside IR35. Contract. Local Government. Central London Contract Details Length: 12 weeks Day rate: £450 per day (Inside IR35) Location: Central London, 1 day per week on site (Friday), 4 days remote Hours: 37 per week Salt's Local Government recruitment team is supporting a Central London authority with a time-critical assignment for a Database Migration Engineer. This role sits within their Technology and Digital Services function and will lead the conversion of legacy Microsoft Access databases into modern data platforms built on SQL Server and SharePoint. Key Responsibilities Migrate legacy Microsoft Access databases into SQL Server, SharePoint lists, and other modern data environments Lead multi-disciplinary teams to plan, design, and execute structured migration activity Develop migration scripts, test plans, and transformation logic to ensure data quality and integrity Coordinate with business users to validate migrated datasets and assess downstream impacts Identify risks, issues, and dependencies, and drive mitigation plans Produce all required delivery documentation, including plans, status reports, design notes, data models and evidence packs Support the organisation through migration cut-over, user acceptance, testing, and post-deployment validation Required Experience Proven track record delivering database or data migration projects in complex organisations Strong hands on experience with Microsoft Access and SQL Server Experience converting Access databases into SQL Server structures and/or SharePoint lists Ability to reverse engineer existing Access logic, schema, forms, queries, macros, and relationships Familiarity with structured migration approaches, including data mapping, cleansing, and reconciliation Experience working with technical and non technical stakeholders across multiple workstreams Ability to produce clear, precise technical documentation, including designs, plans and end user artefacts Desired Exposure Exposure to Local Government, public sector, or other regulated environments Understanding of governance, change control, and assurance processes Knowledge of SSIS, ETL processes, or wider Microsoft data/Power Platform tooling This assignment forms part of a wider modernisation programme aimed at replacing legacy systems with secure, maintainable environments. You will be joining at a critical point, with scope to directly influence the success and pace of delivery. If you have direct experience migrating Access databases into SQL Server and want to support an important Local Government modernisation initiative, we would like to hear from you.
09/06/2026
Full time
Database & Data Migration Engineer (Access to SQL Server) Inside IR35. Contract. Local Government. Central London Contract Details Length: 12 weeks Day rate: £450 per day (Inside IR35) Location: Central London, 1 day per week on site (Friday), 4 days remote Hours: 37 per week Salt's Local Government recruitment team is supporting a Central London authority with a time-critical assignment for a Database Migration Engineer. This role sits within their Technology and Digital Services function and will lead the conversion of legacy Microsoft Access databases into modern data platforms built on SQL Server and SharePoint. Key Responsibilities Migrate legacy Microsoft Access databases into SQL Server, SharePoint lists, and other modern data environments Lead multi-disciplinary teams to plan, design, and execute structured migration activity Develop migration scripts, test plans, and transformation logic to ensure data quality and integrity Coordinate with business users to validate migrated datasets and assess downstream impacts Identify risks, issues, and dependencies, and drive mitigation plans Produce all required delivery documentation, including plans, status reports, design notes, data models and evidence packs Support the organisation through migration cut-over, user acceptance, testing, and post-deployment validation Required Experience Proven track record delivering database or data migration projects in complex organisations Strong hands on experience with Microsoft Access and SQL Server Experience converting Access databases into SQL Server structures and/or SharePoint lists Ability to reverse engineer existing Access logic, schema, forms, queries, macros, and relationships Familiarity with structured migration approaches, including data mapping, cleansing, and reconciliation Experience working with technical and non technical stakeholders across multiple workstreams Ability to produce clear, precise technical documentation, including designs, plans and end user artefacts Desired Exposure Exposure to Local Government, public sector, or other regulated environments Understanding of governance, change control, and assurance processes Knowledge of SSIS, ETL processes, or wider Microsoft data/Power Platform tooling This assignment forms part of a wider modernisation programme aimed at replacing legacy systems with secure, maintainable environments. You will be joining at a critical point, with scope to directly influence the success and pace of delivery. If you have direct experience migrating Access databases into SQL Server and want to support an important Local Government modernisation initiative, we would like to hear from you.
CoreWeave is The Essential Cloud for AI . Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. We're proud to be a Living Wage accredited Employer. What You'll Do: The Monolith AI Platform Engineering Team at CoreWeave is responsible for building and scaling the data and workflow backbone that powers the world's most advanced engineering simulation and AI workflows - our ambition is to become the super intelligent AI test lab for the engineering industry, helping customers ship science, faster. From high throughput data ingestion and feature pipelines to model training and real time inference, our platform delivers the performant, reliable, and trustworthy data foundation trusted by the world's largest engineering companies. The Staff Data Engineer will own and evolve Monolith's platform data services and ETL offerings - the data onboarding, preparation, and lineage capabilities that turn fragmented, real world engineering data into production ready training and inference pipelines. You'll partner with Product, Engineering, and Customer facing teams to deeply understand client data challenges and translate them into scalable, self serve data platform features. About the Role: We're seeking a Staff Data Engineer who can own Monolith's data platform surface end to end: from offline batch pipelines and large historical backfills to low latency, real time streaming data flows that power online inference and feedback loops. You'll define and drive our data architecture, champion data quality and lineage, and decide how customer data moves through Monolith from raw ingestion to governed, observable, and reproducible training sets. You'll primarily work with internal teams (Product, Customer Success, Forward Deployed Engineers, Software Engineers, Data Scientists), and step in as a domain expert when clients need deeper guidance. In this role, you will: Own Monolith's Data Platform & ETL Surface Lead the architecture and evolution of core data services for ingestion, transformation, validation, and lineage across training and inference workloads. Design and maintain end to end data models and schemas that make complex engineering, simulation, and telemetry data discoverable, reusable, and performant. Define standards, contracts, and APIs for how product teams and integrations interact with data services. Design & Operate Batch + Streaming Pipelines Build and operate batch pipelines for large scale historical imports, retraining data sets, and migrations from legacy environments. Design and implement streaming pipelines (e.g., using Kafka or similar technologies) for event driven or real time ingestion and transformation that support online inference, monitoring, and feedback loops. Select and integrate off the shelf, industry proven ETL / ELT technologies and own their rollout and long term operation. Champion Data Lineage, Governance & DataOps Implement and maintain end to end data lineage from source systems to derived features and model artifacts, enabling reproducibility, compliance, and faster debugging. Establish DataOps practices: CI/CD for pipelines, observability (metrics, logs, traces), and operational runbooks for data incidents. Help define data quality and governance standards in partnership with Security, Compliance, and Customer Success, including support for privacy and regulatory needs. Partner Across Monolith & CoreWeave Work with Monolith product and engineering teams to expose data services that unlock new user workflows and AI capabilities. Collaborate with CoreWeave infrastructure and AI platform teams to leverage storage, compute, and observability for reliable data flows. Serve as a technical escalation point for forward deployed and customer facing engineers when questions go deeper than playbooks. Who You Are: Experience & Level 8+ years as a Data Engineer / Data Platform Engineer (or similar), owning production data pipelines and architectural decisions. Staff level impact: leading critical data domains and cross team initiatives. Data Engineering & Architecture Deep experience designing end to end data architectures covering ingestion, storage, transformation, serving, and observability. Hands on experience with both batch and streaming pipelines: batch (historical backfills, retraining) and streaming (Kafka or similar, low latency consumption). Proficiency with SQL and at least one major analytical database or data warehouse (PostgreSQL or similar), including schema design and performance tuning. Proficiency with Spark or Ray or similar distributed data processing frameworks. Solid understanding of data modeling (event logs, star schemas, feature tables) in multi tenant SaaS or platform contexts. Tooling & Ecosystem Hands on with data orchestration and ETL tooling (Airflow, dbt, Dagster, Temporal, or equivalents) and able to evaluate and recommend tools that fit our needs. Experience integrating and operating off the shelf data infrastructure, including rollout, migration plans, and ongoing ownership. Familiarity with cloud infrastructure and containerization (Docker, Kubernetes, at least one major cloud provider) for deploying and scaling data workloads. Data Lineage, Quality & DataOps Extensive experience implementing data lineage solutions for debugging, compliance, and auditability. Strong background in data quality: validation frameworks, monitoring, and guardrails that prevent bad data downstream. Proficiency with DevOps / DataOps practices: infra as code, CI/CD for pipelines, runbooks, on call incident response for data issues. Programming, Systems & Communication Strong programming skills in Python for building data services, transformations, and platform integrations. Comfortable working in service oriented architectures, reasoning about data contracts, SLAs, and failure modes across services. Clear written and verbal communicator able to explain data architectures and trade offs to internal stakeholders and occasionally client conversations. Preferred Experience in ML/AI platforms or MLOps environments where data pipelines feed experimentation, training, and inference workflows at scale. Background with time series, simulation, or experimental data (physical test benches, sensors, engineering simulations). Familiarity with feature stores, experiment tracking systems, or model registries and their integration with upstream pipelines. Experience designing data systems for regulated or safety critical domains, including privacy, residency, and retention considerations. What We Offer Family level Medical Insurance Family level Dental Insurance Generous Pension Contribution Life Assurance at 4x Salary Critical Illness Cover Employee Assistance Programme Tuition Reimbursement Work culture focused on innovative disruption Our Workplace While we prioritize a hybrid work environment, remote work may be considered for candidates located more than 30 miles from an office, based on role requirements for specialized skill sets. New hires will be invited to attend onboarding at one of our hubs within their first month. Teams also gather quarterly to support collaboration. CoreWeave is an equal opportunity employer, committed to fostering an inclusive and supportive workplace. All qualified applicants and candidates will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information. To fulfil our obligation to protect client data, successful applicants offered employment with CoreWeave will be required to complete a basic criminal record check, conducted in compliance with GDPR. Employment offers are conditional upon receiving satisfactory check results.
08/06/2026
Full time
CoreWeave is The Essential Cloud for AI . Built for pioneers by pioneers, CoreWeave delivers a platform of technology, tools, and teams that enables innovators to build and scale AI with confidence. Trusted by leading AI labs, startups, and global enterprises, CoreWeave combines superior infrastructure performance with deep technical expertise to accelerate breakthroughs and turn compute into capability. Founded in 2017, CoreWeave became a publicly traded company (Nasdaq: CRWV) in March 2025. We're proud to be a Living Wage accredited Employer. What You'll Do: The Monolith AI Platform Engineering Team at CoreWeave is responsible for building and scaling the data and workflow backbone that powers the world's most advanced engineering simulation and AI workflows - our ambition is to become the super intelligent AI test lab for the engineering industry, helping customers ship science, faster. From high throughput data ingestion and feature pipelines to model training and real time inference, our platform delivers the performant, reliable, and trustworthy data foundation trusted by the world's largest engineering companies. The Staff Data Engineer will own and evolve Monolith's platform data services and ETL offerings - the data onboarding, preparation, and lineage capabilities that turn fragmented, real world engineering data into production ready training and inference pipelines. You'll partner with Product, Engineering, and Customer facing teams to deeply understand client data challenges and translate them into scalable, self serve data platform features. About the Role: We're seeking a Staff Data Engineer who can own Monolith's data platform surface end to end: from offline batch pipelines and large historical backfills to low latency, real time streaming data flows that power online inference and feedback loops. You'll define and drive our data architecture, champion data quality and lineage, and decide how customer data moves through Monolith from raw ingestion to governed, observable, and reproducible training sets. You'll primarily work with internal teams (Product, Customer Success, Forward Deployed Engineers, Software Engineers, Data Scientists), and step in as a domain expert when clients need deeper guidance. In this role, you will: Own Monolith's Data Platform & ETL Surface Lead the architecture and evolution of core data services for ingestion, transformation, validation, and lineage across training and inference workloads. Design and maintain end to end data models and schemas that make complex engineering, simulation, and telemetry data discoverable, reusable, and performant. Define standards, contracts, and APIs for how product teams and integrations interact with data services. Design & Operate Batch + Streaming Pipelines Build and operate batch pipelines for large scale historical imports, retraining data sets, and migrations from legacy environments. Design and implement streaming pipelines (e.g., using Kafka or similar technologies) for event driven or real time ingestion and transformation that support online inference, monitoring, and feedback loops. Select and integrate off the shelf, industry proven ETL / ELT technologies and own their rollout and long term operation. Champion Data Lineage, Governance & DataOps Implement and maintain end to end data lineage from source systems to derived features and model artifacts, enabling reproducibility, compliance, and faster debugging. Establish DataOps practices: CI/CD for pipelines, observability (metrics, logs, traces), and operational runbooks for data incidents. Help define data quality and governance standards in partnership with Security, Compliance, and Customer Success, including support for privacy and regulatory needs. Partner Across Monolith & CoreWeave Work with Monolith product and engineering teams to expose data services that unlock new user workflows and AI capabilities. Collaborate with CoreWeave infrastructure and AI platform teams to leverage storage, compute, and observability for reliable data flows. Serve as a technical escalation point for forward deployed and customer facing engineers when questions go deeper than playbooks. Who You Are: Experience & Level 8+ years as a Data Engineer / Data Platform Engineer (or similar), owning production data pipelines and architectural decisions. Staff level impact: leading critical data domains and cross team initiatives. Data Engineering & Architecture Deep experience designing end to end data architectures covering ingestion, storage, transformation, serving, and observability. Hands on experience with both batch and streaming pipelines: batch (historical backfills, retraining) and streaming (Kafka or similar, low latency consumption). Proficiency with SQL and at least one major analytical database or data warehouse (PostgreSQL or similar), including schema design and performance tuning. Proficiency with Spark or Ray or similar distributed data processing frameworks. Solid understanding of data modeling (event logs, star schemas, feature tables) in multi tenant SaaS or platform contexts. Tooling & Ecosystem Hands on with data orchestration and ETL tooling (Airflow, dbt, Dagster, Temporal, or equivalents) and able to evaluate and recommend tools that fit our needs. Experience integrating and operating off the shelf data infrastructure, including rollout, migration plans, and ongoing ownership. Familiarity with cloud infrastructure and containerization (Docker, Kubernetes, at least one major cloud provider) for deploying and scaling data workloads. Data Lineage, Quality & DataOps Extensive experience implementing data lineage solutions for debugging, compliance, and auditability. Strong background in data quality: validation frameworks, monitoring, and guardrails that prevent bad data downstream. Proficiency with DevOps / DataOps practices: infra as code, CI/CD for pipelines, runbooks, on call incident response for data issues. Programming, Systems & Communication Strong programming skills in Python for building data services, transformations, and platform integrations. Comfortable working in service oriented architectures, reasoning about data contracts, SLAs, and failure modes across services. Clear written and verbal communicator able to explain data architectures and trade offs to internal stakeholders and occasionally client conversations. Preferred Experience in ML/AI platforms or MLOps environments where data pipelines feed experimentation, training, and inference workflows at scale. Background with time series, simulation, or experimental data (physical test benches, sensors, engineering simulations). Familiarity with feature stores, experiment tracking systems, or model registries and their integration with upstream pipelines. Experience designing data systems for regulated or safety critical domains, including privacy, residency, and retention considerations. What We Offer Family level Medical Insurance Family level Dental Insurance Generous Pension Contribution Life Assurance at 4x Salary Critical Illness Cover Employee Assistance Programme Tuition Reimbursement Work culture focused on innovative disruption Our Workplace While we prioritize a hybrid work environment, remote work may be considered for candidates located more than 30 miles from an office, based on role requirements for specialized skill sets. New hires will be invited to attend onboarding at one of our hubs within their first month. Teams also gather quarterly to support collaboration. CoreWeave is an equal opportunity employer, committed to fostering an inclusive and supportive workplace. All qualified applicants and candidates will receive consideration for employment without regard to race, color, religion, sex, disability, age, sexual orientation, gender identity, national origin, veteran status, or genetic information. To fulfil our obligation to protect client data, successful applicants offered employment with CoreWeave will be required to complete a basic criminal record check, conducted in compliance with GDPR. Employment offers are conditional upon receiving satisfactory check results.
Senior Data Engineer (GCP) Company: Arj Global Location: Manchester, UK Work Model: Hybrid - 80% Remote / 20% On-site (Client Site or Arj Global Office - Trafford) About the Role Arj Global is seeking a highly skilled Senior Data Engineer with over 8 years of experience in data engineering, including at least 3 years of hands on experience with Google Cloud Platform (GCP). The successful candidate will be responsible for designing, building, and optimizing scalable data solutions that support analytics, reporting, and business intelligence initiatives across multiple client engagements. This role requires strong technical expertise, leadership capabilities, and the ability to collaborate with cross functional teams in a hybrid working environment. Key Responsibilities Design, develop, and maintain scalable and reliable data pipelines and ETL/ELT processes. Build and optimize cloud based data platforms using GCP services. Develop data integration solutions for structured and unstructured data sources. Implement data quality, governance, security, and monitoring frameworks. Create and maintain data models to support analytics and reporting requirements. Work closely with business stakeholders, data analysts, architects, and developers to understand data needs and deliver solutions. Optimize performance, scalability, and cost efficiency of data platforms. Mentor junior and mid level data engineers and contribute to engineering best practices. Support data migration and modernization initiatives from on premise or legacy systems to cloud platforms. Participate in solution design, code reviews, and technical documentation. Required Experience & Skills Essential 8+ years of experience in Data Engineering, Data Warehousing, or related fields. 3+ years of hands on experience with Google Cloud Platform (GCP). Strong experience building data pipelines using: Google Cloud Dataflow BigQuery Cloud Storage Pub/Sub Cloud Composer (Apache Airflow) Dataproc Strong SQL development and query optimization skills. Experience with Python and/or Java for data engineering solutions. Experience designing and implementing ETL/ELT frameworks. Strong understanding of data modelling concepts (star schema, dimensional modelling, data vault, etc.). Experience working with large scale datasets and distributed processing frameworks. Knowledge of CI/CD practices and DevOps principles. Experience with Git based version control systems. Strong problem solving, analytical, and communication skills. Preferred Experience with Apache Spark. Experience with Terraform or Infrastructure as Code (IaC). Knowledge of data governance and data security best practices. Experience working in Agile/Scrum environments. GCP Professional Data Engineer certification. Exposure to multi cloud environments (AWS and/or Azure). Leadership & Soft Skills Ability to lead technical discussions and influence architectural decisions. Strong stakeholder management skills. Experience mentoring and coaching engineering teams. Excellent communication skills with both technical and non technical audiences. Ability to work independently in a hybrid environment. Education Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field. Relevant cloud and data engineering certifications are highly desirable. Working Arrangement Location: Manchester, UK Remote Working: 80% On site Requirement: 20% (at client locations and/or Arj Global offices as required) Occasional travel may be required based on project needs. What We Offer Competitive salary and benefits package. Flexible hybrid working model. Opportunity to work on enterprise scale cloud and data transformation projects. Professional development and certification support. Collaborative and innovative working environment. Experience Required: 8+ years Data Engineering, including 3+ years GCP Seniority Level: Senior Data Engineer Employment Type: Full time, Permanent
02/06/2026
Full time
Senior Data Engineer (GCP) Company: Arj Global Location: Manchester, UK Work Model: Hybrid - 80% Remote / 20% On-site (Client Site or Arj Global Office - Trafford) About the Role Arj Global is seeking a highly skilled Senior Data Engineer with over 8 years of experience in data engineering, including at least 3 years of hands on experience with Google Cloud Platform (GCP). The successful candidate will be responsible for designing, building, and optimizing scalable data solutions that support analytics, reporting, and business intelligence initiatives across multiple client engagements. This role requires strong technical expertise, leadership capabilities, and the ability to collaborate with cross functional teams in a hybrid working environment. Key Responsibilities Design, develop, and maintain scalable and reliable data pipelines and ETL/ELT processes. Build and optimize cloud based data platforms using GCP services. Develop data integration solutions for structured and unstructured data sources. Implement data quality, governance, security, and monitoring frameworks. Create and maintain data models to support analytics and reporting requirements. Work closely with business stakeholders, data analysts, architects, and developers to understand data needs and deliver solutions. Optimize performance, scalability, and cost efficiency of data platforms. Mentor junior and mid level data engineers and contribute to engineering best practices. Support data migration and modernization initiatives from on premise or legacy systems to cloud platforms. Participate in solution design, code reviews, and technical documentation. Required Experience & Skills Essential 8+ years of experience in Data Engineering, Data Warehousing, or related fields. 3+ years of hands on experience with Google Cloud Platform (GCP). Strong experience building data pipelines using: Google Cloud Dataflow BigQuery Cloud Storage Pub/Sub Cloud Composer (Apache Airflow) Dataproc Strong SQL development and query optimization skills. Experience with Python and/or Java for data engineering solutions. Experience designing and implementing ETL/ELT frameworks. Strong understanding of data modelling concepts (star schema, dimensional modelling, data vault, etc.). Experience working with large scale datasets and distributed processing frameworks. Knowledge of CI/CD practices and DevOps principles. Experience with Git based version control systems. Strong problem solving, analytical, and communication skills. Preferred Experience with Apache Spark. Experience with Terraform or Infrastructure as Code (IaC). Knowledge of data governance and data security best practices. Experience working in Agile/Scrum environments. GCP Professional Data Engineer certification. Exposure to multi cloud environments (AWS and/or Azure). Leadership & Soft Skills Ability to lead technical discussions and influence architectural decisions. Strong stakeholder management skills. Experience mentoring and coaching engineering teams. Excellent communication skills with both technical and non technical audiences. Ability to work independently in a hybrid environment. Education Bachelor's degree in Computer Science, Engineering, Information Systems, or a related field. Relevant cloud and data engineering certifications are highly desirable. Working Arrangement Location: Manchester, UK Remote Working: 80% On site Requirement: 20% (at client locations and/or Arj Global offices as required) Occasional travel may be required based on project needs. What We Offer Competitive salary and benefits package. Flexible hybrid working model. Opportunity to work on enterprise scale cloud and data transformation projects. Professional development and certification support. Collaborative and innovative working environment. Experience Required: 8+ years Data Engineering, including 3+ years GCP Seniority Level: Senior Data Engineer Employment Type: Full time, Permanent
Hours: 40 pw (9.30-6pm Monday to Friday, 1 hour lunch) Term: FTC - 6 months Location: UK or US (Remote) About us Artlogic is a leading technology platform for the art world. Founded in 1989, Artlogic empowers over 6,000 galleries, dealers, and art professionals worldwide to streamline their operations and grow their businesses. The company's integrated platform combines inventory management, CRM, websites, sales, marketing, and payment tools, everything needed to run a modern and successful art business. In 2021, Artlogic was acquired by tech investors Cove Hill Partners to accelerate innovationand expand its global reach. Since then, the company has doubled in size, with employees across London, New York, Atlanta, and Cape Town. Today, Artlogic is entering an exciting new chapter through its merger with ArtCloud, another industry leader in art technology. Together, the two companies are uniting complementary platforms to build the most comprehensive, forward-thinking solution for the art world. This collaboration brings together deep expertise across CRM, sales, marketing, inventory, payments, and AI-powered tools, empowering galleries, artists, and collectors worldwide. Joining Artlogic means becoming part of a growing global team that is shaping the future of how art is bought, sold, and managed, both online and in person. The Data Migration team operates within the Customer function, combining both technical and project delivery responsibilities. Project Managers act as the primary client interface, coordinating communication and requirements. Data Migration Specialists are responsible for technical execution, including ETL processes and solution design. The team also contributes to internal R&D, building tools and applications to improve migration efficiency and scalability. The Role As a Junior Data Engineer, you will be a vital part of the team responsible for the lifeblood of our onboarding process: client data. You will help bridge the gap between legacy client systems and the Artlogic ecosystem by building and executing robust ETL (Extract, Transform, Load) pipelines. This role is ideal for a developer early in their career who loves the "puzzle" aspect of data. You will work within the Data Migration team to dismantle complex external data structures (Relational data, FileMaker), reassemble them into our sophisticated data model, and contribute to the internal Python/Ruby libraries that make our migrations faster and smarter. You will be mentored by senior engineers and work closely with Project Managers to ensure a seamless transition for our clients. Key Responsibilities ETL Development & Execution Execute End-to-End Migrations: Learn to navigate the full migration lifecycle, moving data from diverse client environments into Artlogic. Extraction: Use scripts and tools to pull data from external platforms (e.g., FileMaker, legacy SQL databases). Transformation: Write clean, maintainable code to map, clean, and transform "messy" legacy data to fit the Artlogic schema. QA & Validation: Perform rigorous data validation and unit testing to ensure 100% accuracy and data integrity before go live. Technical Growth & Tooling Code Contribution: Assist in developing and maintaining our internal migration frameworks and R&D tools. Pipeline Optimization: Identify repetitive tasks in your daily workflow and work with senior engineers to automate them. Systems Analysis: Research unknown database structures to determine the most efficient extraction methods. Cross-functional Teamwork: Partner with Client Liaison Project Managers to understand client-specific data nuances. Documentation: Document migration mappings and technical edge cases to ensure knowledge is shared across the team. Iterative Delivery: Adapt migration scripts quickly as project requirements evolve during the onboarding phase. AI & Data AI Recommendation Verification and Data Mapping: Review and verify AI (Claude) recommendations for column mappings during cloud data migration, including confirming that columns fit perfectly or identifying where data requires manipulation or concatenation. Data Transformation and Cloud Push: Perform final data transformation steps using basic Python, Pandas, and Reax code to prepare and push relational data from Excel into the cloud environment. Key Attributes Data Integrity: Success is measured by the accuracy of migrated data and the absence of "broken" records post launch. Technical Progression: Demonstrates a growing ability to handle increasingly complex data structures with less supervision. Code Quality: Contributions to internal libraries follow team standards and improve overall migration speed. Reliability: Consistently meets project milestones and keeps stakeholders informed of technical blockers. About You Coding Foundations: Proficiency in at least one object-oriented language-Python is preferred, but we also value strong logic in C#. SQL & Data Logic: Solid understanding of relational databases (Joins, Keys, Indexes) and how to manipulate data structures. Problem-Solving Mindset: You enjoy the "detective work" of figuring out how a legacy system was built and how to extract its value. Detail Oriented: An obsession with data accuracy; you catch the "edge case" before it becomes a bug. Communication: Ability to explain technical data hurdles to non-technical Project Managers. The "Data Stack": Experience with ETL tools, pandas (Python), or regular expressions (Regex). API Knowledge: Familiarity with REST APIs and how to handle JSON/XML data. Niche Systems: Previous exposure to FileMaker (Claris) or NoSQL environments. Web Basics: A high-level understanding of how back end data interacts with front end UIs. Artlogic is an equal opportunities employer Artlogic is an equal opportunities employer. We are committed to fair and inclusive recruitment and employment practices. We welcome applications from all suitably qualified candidates. We are committed to making reasonable adjustments for disabled applicants and employees. If you have a disability and would like us to make adjustments to the application or interview process, or to discuss any adjustments to the role, please let us know. Please note that this position is UK or US based. Candidates must therefore have, or be able to obtain, appropriate immigration status entitling them to work in the UK or US for us in the role. Please note that we do not have a sponsor licence and we are not able to offer visa sponsorship for this position. The successful candidate will need to provide evidence/information to enable us to verify their right to work, prior to commencement of employment, and in accordance with current Home Office right to work guidance.
30/05/2026
Full time
Hours: 40 pw (9.30-6pm Monday to Friday, 1 hour lunch) Term: FTC - 6 months Location: UK or US (Remote) About us Artlogic is a leading technology platform for the art world. Founded in 1989, Artlogic empowers over 6,000 galleries, dealers, and art professionals worldwide to streamline their operations and grow their businesses. The company's integrated platform combines inventory management, CRM, websites, sales, marketing, and payment tools, everything needed to run a modern and successful art business. In 2021, Artlogic was acquired by tech investors Cove Hill Partners to accelerate innovationand expand its global reach. Since then, the company has doubled in size, with employees across London, New York, Atlanta, and Cape Town. Today, Artlogic is entering an exciting new chapter through its merger with ArtCloud, another industry leader in art technology. Together, the two companies are uniting complementary platforms to build the most comprehensive, forward-thinking solution for the art world. This collaboration brings together deep expertise across CRM, sales, marketing, inventory, payments, and AI-powered tools, empowering galleries, artists, and collectors worldwide. Joining Artlogic means becoming part of a growing global team that is shaping the future of how art is bought, sold, and managed, both online and in person. The Data Migration team operates within the Customer function, combining both technical and project delivery responsibilities. Project Managers act as the primary client interface, coordinating communication and requirements. Data Migration Specialists are responsible for technical execution, including ETL processes and solution design. The team also contributes to internal R&D, building tools and applications to improve migration efficiency and scalability. The Role As a Junior Data Engineer, you will be a vital part of the team responsible for the lifeblood of our onboarding process: client data. You will help bridge the gap between legacy client systems and the Artlogic ecosystem by building and executing robust ETL (Extract, Transform, Load) pipelines. This role is ideal for a developer early in their career who loves the "puzzle" aspect of data. You will work within the Data Migration team to dismantle complex external data structures (Relational data, FileMaker), reassemble them into our sophisticated data model, and contribute to the internal Python/Ruby libraries that make our migrations faster and smarter. You will be mentored by senior engineers and work closely with Project Managers to ensure a seamless transition for our clients. Key Responsibilities ETL Development & Execution Execute End-to-End Migrations: Learn to navigate the full migration lifecycle, moving data from diverse client environments into Artlogic. Extraction: Use scripts and tools to pull data from external platforms (e.g., FileMaker, legacy SQL databases). Transformation: Write clean, maintainable code to map, clean, and transform "messy" legacy data to fit the Artlogic schema. QA & Validation: Perform rigorous data validation and unit testing to ensure 100% accuracy and data integrity before go live. Technical Growth & Tooling Code Contribution: Assist in developing and maintaining our internal migration frameworks and R&D tools. Pipeline Optimization: Identify repetitive tasks in your daily workflow and work with senior engineers to automate them. Systems Analysis: Research unknown database structures to determine the most efficient extraction methods. Cross-functional Teamwork: Partner with Client Liaison Project Managers to understand client-specific data nuances. Documentation: Document migration mappings and technical edge cases to ensure knowledge is shared across the team. Iterative Delivery: Adapt migration scripts quickly as project requirements evolve during the onboarding phase. AI & Data AI Recommendation Verification and Data Mapping: Review and verify AI (Claude) recommendations for column mappings during cloud data migration, including confirming that columns fit perfectly or identifying where data requires manipulation or concatenation. Data Transformation and Cloud Push: Perform final data transformation steps using basic Python, Pandas, and Reax code to prepare and push relational data from Excel into the cloud environment. Key Attributes Data Integrity: Success is measured by the accuracy of migrated data and the absence of "broken" records post launch. Technical Progression: Demonstrates a growing ability to handle increasingly complex data structures with less supervision. Code Quality: Contributions to internal libraries follow team standards and improve overall migration speed. Reliability: Consistently meets project milestones and keeps stakeholders informed of technical blockers. About You Coding Foundations: Proficiency in at least one object-oriented language-Python is preferred, but we also value strong logic in C#. SQL & Data Logic: Solid understanding of relational databases (Joins, Keys, Indexes) and how to manipulate data structures. Problem-Solving Mindset: You enjoy the "detective work" of figuring out how a legacy system was built and how to extract its value. Detail Oriented: An obsession with data accuracy; you catch the "edge case" before it becomes a bug. Communication: Ability to explain technical data hurdles to non-technical Project Managers. The "Data Stack": Experience with ETL tools, pandas (Python), or regular expressions (Regex). API Knowledge: Familiarity with REST APIs and how to handle JSON/XML data. Niche Systems: Previous exposure to FileMaker (Claris) or NoSQL environments. Web Basics: A high-level understanding of how back end data interacts with front end UIs. Artlogic is an equal opportunities employer Artlogic is an equal opportunities employer. We are committed to fair and inclusive recruitment and employment practices. We welcome applications from all suitably qualified candidates. We are committed to making reasonable adjustments for disabled applicants and employees. If you have a disability and would like us to make adjustments to the application or interview process, or to discuss any adjustments to the role, please let us know. Please note that this position is UK or US based. Candidates must therefore have, or be able to obtain, appropriate immigration status entitling them to work in the UK or US for us in the role. Please note that we do not have a sponsor licence and we are not able to offer visa sponsorship for this position. The successful candidate will need to provide evidence/information to enable us to verify their right to work, prior to commencement of employment, and in accordance with current Home Office right to work guidance.
Purpose of the Job: Design, build, and maintain robust data systems and pipelines that support data storage, processing, and analysis on the Cloud. Work with large datasets, ensuring data quality, scalability, and performance, while collaborating. closely with data scientists, analysts, and other engineering teams to understand their data needs and provide them with high-quality, accessible data. They are responsible for ensuring that the underlying data infrastructure supports the organizations broader data and business goals, enabling more effective data-driven decision-making. Key Accountabilities: Design and implement scalable, efficient, and secure data architectures, ensuring optimal data flow across systems in order to achieve high service levels of support, maintenance and development You will own development and change projects to ensure requirements are met in the most cost-effective manner while minimising associated risk to expected standards. Responsibly for cloud data platform development, data modelling, shaping and technical planning You will be a mentor among the owning decision making and evaluation of requirement suitability, facilitate reliable estimates, technical project management, stakeholder management with a project Ensure that resource requirements are understood and planned/estimated effectively against demand, including identification of additional temporary resource capability within projects Maintain appropriate process procedures, compliance and service level monitoring, performance reporting and vendor management. Implementing best practices around data security, privacy, and compliance for the teams compliance with cyber security and data protection and supporting along with BI lead Strong stakeholder management will be required for maintaining relationships with our business users to clarify and influence requirements. Including liaising with internal business departments and functions to manage the service level expected from the data team. Collaborating with external organisations and third-party software/service suppliers for ongoing support, maintenance and development of systems. You will be able to demonstrate you are quality focused to ensure that they solutions are built to an appropriate standard whilst being balanced with a drive to deliver against tight deadlines. Support in developing and implementing best practices and process across the team along with BI lead. Influence the evolution of business and system requirements and contribute to the design of technical solutions to feed a delivery pipeline that increasingly employs Agile methods such as SCRUM and Kanban You will be required to develop unit tested code and then support test cycles including post implementation validation. You will be required to contribute to the transition into service and ongoing support of the applications in the area which provides the opportunity to reduce technical debt and rationalise our technical footprint Mentor data engineers, supporting their professional growth and development Outcome, Results and Key Performance Indicators: Delivery of projects to expected timely, cost and quality standards Excellent levels of application availability and resilience as required by business operations. Necessary governance and control requirements defined - design, code and test standards and guidelines. Ensure data systems comply with necessary governance and control requirements. Internally-developed data solutions are fit for purpose and fit correctly within the data architecture. Built and tested to user requirements, performing to defined performance and capacity requirements. Company data is secure, accurate, maintained and available according to requirements. Technical risks and issues correctly mitigated and managed on Projects and Production support. High quality software delivered in to production - zero critical and high defects before production release. Dimensions of Job: This role is part of a well-established data team, the role offers a great opportunity for the right candidate to hone their modern data management skills in a friendly and supportive environment. This role requires attendance to a Leeds based office as often as needed with a minimum 2 days a week. Able to work effectively as part of a remote team. A great opportunity for a motivated data engineer seeking a new opportunity with a friendly, newly formed data team and able to contribute to the team's growth with their technical expertise Key Relationships: Internal: Wider technical teams (including apps, test, dev ops and more), Project managers, business SME's, data teams and communities , Data scientists, BI Lead, Head of Data External: software & service suppliers, consultants. Knowledge and Skills: Knowledge - Broad data management technical knowledge so as to be able to work across full data cycle. - Proven Experience working with AWS data technologies (S3, Redshift, Glue, Lambda, Lake formation, Cloud Formation), GitHub, CI/CD - Coding experience in Apache Spark, Iceberg or Python (Pandas) - Experience in change and release management. - Experience in Database Warehouse design and data modelling - Experience managing Data Migration projects. - Cloud data platform development and deployment. - Experience of performance tuning in a variery of database settings. - Experience of Infrastructure as code practises. - Proven ability to organise and produce work within deadlines. Skills - Good project and people management skills. - Excellent data development skills. - Excellent data manipulation and analysis skills using a variety of tools including SQL, Phyton, AWS services and the MSBI stack. - Ability to prioritise and be flexible to change those priorities at short notice. - Commercial acumen. - Able to demonstrate a practical approach to problem solving. - Able to provide appropriate and understandable data to a wide ranging audience. - Well-developed and professional communication skills. - Strong analytical skills - ability to create models and analyse data in order to solve complex problems or reinforce commercial decisions. - Able to understand business processes and how this is achieved/influenced by technology. - Must be able to work as part of a collaborative team to solve problems and assist other colleagues. - Ability to learn new technologies, programs and procedures. Technical Essentials: - Expertise across data warehouse and ETL/ ELT development in AWS preferred with experience in the following: - Strong experience in some of the AWS services like Redshift, Lambda,S3,Step Functions, Batch, Cloud formation, Lake Formation, Code Build, CI/CD, GitHub, IAM, SQS, SNS, Aurora DB - Good experience with DBT, Apache Iceberg, Docker, Microsoft BI stack (nice to have) - Experience in data warehouse design (Kimball and lake house, medallion and data vault) is a definite preference as is knowledge of other data tools and programming languages such as Python & Spark and Strong SQL experience. - Experience is building Data lake and building CI/CD data pipelines - A candidate is expected to understand and can demonstrate experience across the delivery lifecycle and understand both Agile and Waterfall methods and when to apply these. Experience: This position requires several years of practical experience in a similar environment. We require a good balance of technical and personal/softer skills so successful candidates can be fully effective immediately. - Proven experience in developing, delivering and maintaining tactical and enterprise data management solutions. - Proven experience in delivering data solutions using cloud platform tools. - Proven experience in assessing the impact of proposed changes on production solutions. - Proven experience in managing and developing a team of technical experts to deliver business outcomes and meet performance criteria. - Exposure to Energy markets, Energy Supply industry sector - Developing and implementing operational processes and procedures.
02/10/2025
Full time
Purpose of the Job: Design, build, and maintain robust data systems and pipelines that support data storage, processing, and analysis on the Cloud. Work with large datasets, ensuring data quality, scalability, and performance, while collaborating. closely with data scientists, analysts, and other engineering teams to understand their data needs and provide them with high-quality, accessible data. They are responsible for ensuring that the underlying data infrastructure supports the organizations broader data and business goals, enabling more effective data-driven decision-making. Key Accountabilities: Design and implement scalable, efficient, and secure data architectures, ensuring optimal data flow across systems in order to achieve high service levels of support, maintenance and development You will own development and change projects to ensure requirements are met in the most cost-effective manner while minimising associated risk to expected standards. Responsibly for cloud data platform development, data modelling, shaping and technical planning You will be a mentor among the owning decision making and evaluation of requirement suitability, facilitate reliable estimates, technical project management, stakeholder management with a project Ensure that resource requirements are understood and planned/estimated effectively against demand, including identification of additional temporary resource capability within projects Maintain appropriate process procedures, compliance and service level monitoring, performance reporting and vendor management. Implementing best practices around data security, privacy, and compliance for the teams compliance with cyber security and data protection and supporting along with BI lead Strong stakeholder management will be required for maintaining relationships with our business users to clarify and influence requirements. Including liaising with internal business departments and functions to manage the service level expected from the data team. Collaborating with external organisations and third-party software/service suppliers for ongoing support, maintenance and development of systems. You will be able to demonstrate you are quality focused to ensure that they solutions are built to an appropriate standard whilst being balanced with a drive to deliver against tight deadlines. Support in developing and implementing best practices and process across the team along with BI lead. Influence the evolution of business and system requirements and contribute to the design of technical solutions to feed a delivery pipeline that increasingly employs Agile methods such as SCRUM and Kanban You will be required to develop unit tested code and then support test cycles including post implementation validation. You will be required to contribute to the transition into service and ongoing support of the applications in the area which provides the opportunity to reduce technical debt and rationalise our technical footprint Mentor data engineers, supporting their professional growth and development Outcome, Results and Key Performance Indicators: Delivery of projects to expected timely, cost and quality standards Excellent levels of application availability and resilience as required by business operations. Necessary governance and control requirements defined - design, code and test standards and guidelines. Ensure data systems comply with necessary governance and control requirements. Internally-developed data solutions are fit for purpose and fit correctly within the data architecture. Built and tested to user requirements, performing to defined performance and capacity requirements. Company data is secure, accurate, maintained and available according to requirements. Technical risks and issues correctly mitigated and managed on Projects and Production support. High quality software delivered in to production - zero critical and high defects before production release. Dimensions of Job: This role is part of a well-established data team, the role offers a great opportunity for the right candidate to hone their modern data management skills in a friendly and supportive environment. This role requires attendance to a Leeds based office as often as needed with a minimum 2 days a week. Able to work effectively as part of a remote team. A great opportunity for a motivated data engineer seeking a new opportunity with a friendly, newly formed data team and able to contribute to the team's growth with their technical expertise Key Relationships: Internal: Wider technical teams (including apps, test, dev ops and more), Project managers, business SME's, data teams and communities , Data scientists, BI Lead, Head of Data External: software & service suppliers, consultants. Knowledge and Skills: Knowledge - Broad data management technical knowledge so as to be able to work across full data cycle. - Proven Experience working with AWS data technologies (S3, Redshift, Glue, Lambda, Lake formation, Cloud Formation), GitHub, CI/CD - Coding experience in Apache Spark, Iceberg or Python (Pandas) - Experience in change and release management. - Experience in Database Warehouse design and data modelling - Experience managing Data Migration projects. - Cloud data platform development and deployment. - Experience of performance tuning in a variery of database settings. - Experience of Infrastructure as code practises. - Proven ability to organise and produce work within deadlines. Skills - Good project and people management skills. - Excellent data development skills. - Excellent data manipulation and analysis skills using a variety of tools including SQL, Phyton, AWS services and the MSBI stack. - Ability to prioritise and be flexible to change those priorities at short notice. - Commercial acumen. - Able to demonstrate a practical approach to problem solving. - Able to provide appropriate and understandable data to a wide ranging audience. - Well-developed and professional communication skills. - Strong analytical skills - ability to create models and analyse data in order to solve complex problems or reinforce commercial decisions. - Able to understand business processes and how this is achieved/influenced by technology. - Must be able to work as part of a collaborative team to solve problems and assist other colleagues. - Ability to learn new technologies, programs and procedures. Technical Essentials: - Expertise across data warehouse and ETL/ ELT development in AWS preferred with experience in the following: - Strong experience in some of the AWS services like Redshift, Lambda,S3,Step Functions, Batch, Cloud formation, Lake Formation, Code Build, CI/CD, GitHub, IAM, SQS, SNS, Aurora DB - Good experience with DBT, Apache Iceberg, Docker, Microsoft BI stack (nice to have) - Experience in data warehouse design (Kimball and lake house, medallion and data vault) is a definite preference as is knowledge of other data tools and programming languages such as Python & Spark and Strong SQL experience. - Experience is building Data lake and building CI/CD data pipelines - A candidate is expected to understand and can demonstrate experience across the delivery lifecycle and understand both Agile and Waterfall methods and when to apply these. Experience: This position requires several years of practical experience in a similar environment. We require a good balance of technical and personal/softer skills so successful candidates can be fully effective immediately. - Proven experience in developing, delivering and maintaining tactical and enterprise data management solutions. - Proven experience in delivering data solutions using cloud platform tools. - Proven experience in assessing the impact of proposed changes on production solutions. - Proven experience in managing and developing a team of technical experts to deliver business outcomes and meet performance criteria. - Exposure to Energy markets, Energy Supply industry sector - Developing and implementing operational processes and procedures.
Burgers & Fries and Incredible Careers! We're the burger restaurant with the uncomplicated formula: burgers and fries cooked to perfection, with no frozen ingredients. And we've stuck to the same 'perfect and serve' philosophy since our family business began in 1986. We are seeking a highly capable and driven Data Engineer to join our dynamic data team. This role is pivotal in leading and initiating the migration of our data infrastructure from AWS to Microsoft Azure, shaping the future state of our data platform. You'll work at the heart of the business, enabling data integration, transformation, and reporting across a range of complex data sources. This is a strategic, hands-on role that requires someone who is not afraid to explore new technologies, drive change, and build future-proof data engineering solutions. You'll collaborate with cross-functional teams, manage data integrations, and optimize the performance of databases and cloud environments in a high-growth, fast-paced setting. YOUR RESPONSIBILITIES: Cloud Migration Leadership Play a key role in the execution of our data migration from AWS to Azure, working closely with the Solution Architect and Head of data to shape the technical approach and deliver scalable, cloud-native solutions. Evaluate existing data infrastructure and propose Azure-native solutions aligned to business needs. Define and lead the migration roadmap, collaborating with internal stakeholders and third parties. Advocate for innovation and drive adoption of Azure services, standards, and tools across the team. Apply and champion Azure Medallion architecture principles in future-state design. Data Ingestion & Integration Design and implement ingestion pipelines into Azure (Data Lake, Synapse, Blob Storage, Azure SQL). Ingest structured, semi-structured, and unstructured data from APIs, FTP/SFTP, cloud platforms, and file-based sources. Manage data integrations with suppliers, including file transfers and API-based solutions. Use Power Automate (Flow) to automate tasks like copying files from SharePoint or emails to SFTP. Provide third-line support for code issues and existing workflows, lead resolution of data failures and integration problems. Collaborate with stakeholders to define integration requirements and implement robust solutions. Data Transformation & ETL Build, optimize, and maintain ETL/ELT pipelines using Azure Data Factory and other cloud-native tools. Develop reusable, scalable transformation logic to enable analytics and reporting needs. Apply data validation, cleansing, and lineage tracking throughout the pipeline lifecycle. Support both existing and future ETL workloads with strong SQL and scripting capabilities. Data Platform & Server Management Revamp and support existing database designs to enable a scalable migration to Azure. Introduce and enforce standards and governance around data structures and code. Optimize T-SQL code and SQL Server performance before deploying into production. Manage consistent data platforms across regions with a focus on reusability and modularity. Monitor and control database file growth, indexing strategies, and performance tuning. Develop scalable solutions based on both current and emerging business requirements. Data Cloud Operations & Resilience Lead initiatives in data cloud management, including usage monitoring and optimization. Configure and test disaster recovery processes, ensuring business continuity through backup and restore exercises. Archive or delete legacy data in line with governance and retention requirements. Delivery & Documentation Conduct data mapping exercises, analysing various sources and formats to design fit-for-purpose models. Create and present complete logical data models to both technical teams and business stakeholders. Participate in scoping future platforms and assessing legacy systems for migration to Azure. Perform functional and systems analysis, producing supporting documentation (e.g., flowcharts, data diagrams). Be the lead in discovery workshops, weekly stand-ups, and cross-functional solution delivery. YOUR TECHNICAL SKILLS & EXPERERIENCE Essential 6-10 years' experience working with SQL Server (including T-SQL development and performance tuning). Proven experience designing ETL/ELT pipelines, ideally in Azure Data Factory. Hands-on experience ingesting and transforming data in cloud environments (preferably Azure). Familiarity with Azure Medallion architecture or similar layered data designs. Integration experience with APIs, SFTP, cloud databases, and file systems. Comfortable working with Power Automate, SharePoint, and Office 365 integrations. Basic knowledge of C# or PowerShell for custom data integration tasks. Experience troubleshooting production systems and optimizing SQL workloads. Strong understanding of database administration, performance, and scalability. Cloud certification (e.g., Azure Practitioner or equivalent) is highly desirable. Desirable Experience working with notebooks and Python in Microsoft Fabric for data exploration, transformation, and pipeline development. Experience with AWS services and data environments, especially helpful for transition planning. Exposure to CI/CD, DevOps pipelines, and Git for data engineering workflows. Familiarity with data governance, quality management, and security practices. YOUR BEHAVIOURS & ATTRIBUTES Technically curious and excited by the opportunity to learn and adopt new tools. Naturally proactive and comfortable driving change in existing environments. Calm and solution-oriented in high-pressure or production-critical scenarios. Collaborative and open communicator who brings people on the journey. Able to travel occasionally within Europe to support cross-territory platforms and teams. LOCATION This role is based in our West London Office including some remote working
02/10/2025
Full time
Burgers & Fries and Incredible Careers! We're the burger restaurant with the uncomplicated formula: burgers and fries cooked to perfection, with no frozen ingredients. And we've stuck to the same 'perfect and serve' philosophy since our family business began in 1986. We are seeking a highly capable and driven Data Engineer to join our dynamic data team. This role is pivotal in leading and initiating the migration of our data infrastructure from AWS to Microsoft Azure, shaping the future state of our data platform. You'll work at the heart of the business, enabling data integration, transformation, and reporting across a range of complex data sources. This is a strategic, hands-on role that requires someone who is not afraid to explore new technologies, drive change, and build future-proof data engineering solutions. You'll collaborate with cross-functional teams, manage data integrations, and optimize the performance of databases and cloud environments in a high-growth, fast-paced setting. YOUR RESPONSIBILITIES: Cloud Migration Leadership Play a key role in the execution of our data migration from AWS to Azure, working closely with the Solution Architect and Head of data to shape the technical approach and deliver scalable, cloud-native solutions. Evaluate existing data infrastructure and propose Azure-native solutions aligned to business needs. Define and lead the migration roadmap, collaborating with internal stakeholders and third parties. Advocate for innovation and drive adoption of Azure services, standards, and tools across the team. Apply and champion Azure Medallion architecture principles in future-state design. Data Ingestion & Integration Design and implement ingestion pipelines into Azure (Data Lake, Synapse, Blob Storage, Azure SQL). Ingest structured, semi-structured, and unstructured data from APIs, FTP/SFTP, cloud platforms, and file-based sources. Manage data integrations with suppliers, including file transfers and API-based solutions. Use Power Automate (Flow) to automate tasks like copying files from SharePoint or emails to SFTP. Provide third-line support for code issues and existing workflows, lead resolution of data failures and integration problems. Collaborate with stakeholders to define integration requirements and implement robust solutions. Data Transformation & ETL Build, optimize, and maintain ETL/ELT pipelines using Azure Data Factory and other cloud-native tools. Develop reusable, scalable transformation logic to enable analytics and reporting needs. Apply data validation, cleansing, and lineage tracking throughout the pipeline lifecycle. Support both existing and future ETL workloads with strong SQL and scripting capabilities. Data Platform & Server Management Revamp and support existing database designs to enable a scalable migration to Azure. Introduce and enforce standards and governance around data structures and code. Optimize T-SQL code and SQL Server performance before deploying into production. Manage consistent data platforms across regions with a focus on reusability and modularity. Monitor and control database file growth, indexing strategies, and performance tuning. Develop scalable solutions based on both current and emerging business requirements. Data Cloud Operations & Resilience Lead initiatives in data cloud management, including usage monitoring and optimization. Configure and test disaster recovery processes, ensuring business continuity through backup and restore exercises. Archive or delete legacy data in line with governance and retention requirements. Delivery & Documentation Conduct data mapping exercises, analysing various sources and formats to design fit-for-purpose models. Create and present complete logical data models to both technical teams and business stakeholders. Participate in scoping future platforms and assessing legacy systems for migration to Azure. Perform functional and systems analysis, producing supporting documentation (e.g., flowcharts, data diagrams). Be the lead in discovery workshops, weekly stand-ups, and cross-functional solution delivery. YOUR TECHNICAL SKILLS & EXPERERIENCE Essential 6-10 years' experience working with SQL Server (including T-SQL development and performance tuning). Proven experience designing ETL/ELT pipelines, ideally in Azure Data Factory. Hands-on experience ingesting and transforming data in cloud environments (preferably Azure). Familiarity with Azure Medallion architecture or similar layered data designs. Integration experience with APIs, SFTP, cloud databases, and file systems. Comfortable working with Power Automate, SharePoint, and Office 365 integrations. Basic knowledge of C# or PowerShell for custom data integration tasks. Experience troubleshooting production systems and optimizing SQL workloads. Strong understanding of database administration, performance, and scalability. Cloud certification (e.g., Azure Practitioner or equivalent) is highly desirable. Desirable Experience working with notebooks and Python in Microsoft Fabric for data exploration, transformation, and pipeline development. Experience with AWS services and data environments, especially helpful for transition planning. Exposure to CI/CD, DevOps pipelines, and Git for data engineering workflows. Familiarity with data governance, quality management, and security practices. YOUR BEHAVIOURS & ATTRIBUTES Technically curious and excited by the opportunity to learn and adopt new tools. Naturally proactive and comfortable driving change in existing environments. Calm and solution-oriented in high-pressure or production-critical scenarios. Collaborative and open communicator who brings people on the journey. Able to travel occasionally within Europe to support cross-territory platforms and teams. LOCATION This role is based in our West London Office including some remote working
Title- Cloud Data Engineer Location- Remote Lemongrass Consulting is the leading professional and managed service provider of SAP enterprise applications running on AWS hyperscale cloud infrastructure. Our objective is to delight our customers every day by reducing the cost and increasing the agility of their SAP systems. We do this with our continuous innovation, automation, migration and operation, delivered on the world's most comprehensive cloud platforms. Our team is what makes Lemongrass exceptional and why we have the excellent reputation in the market that we enjoy today. At Lemongrass, you will work with the smartest and most motivated people in the business. We take pride in our culture of innovation and collaboration that drives us to deliver exceptional benefits to our clients every day. About the Role: We are seeking an experienced Cloud Data Engineer with a strong background in AWS, Azure, and GCP. The ideal candidate will have extensive experience with cloud-native ETL tools such as AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, and other ETL tools like Informatica, SAP Data Intelligence, etc. You will be responsible for designing, implementing, and maintaining robust data pipelines and building scalable data lakes. Experience with various data platforms like Redshift, Snowflake, Databricks, Synapse, Snowflake and others is essential. Familiarity with data extraction from SAP or ERP systems is a plus. Key Responsibilities: Design and Development: Design, develop, and maintain scalable ETL pipelines using cloud-native tools (AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, etc.). Architect and implement data lakes and data warehouses on cloud platforms (AWS, Azure, GCP). Develop and optimize data ingestion, transformation, and loading processes using Databricks, Snowflake, Redshift, BigQuery and Azure Synapse. Implement ETL processes using tools like Informatica, SAP Data Intelligence, and others. Develop and optimize data processing jobs using Spark Scala. Data Integration and Management: Integrate various data sources, including relational databases, APIs, unstructured data, and ERP systems into the data lake. Ensure data quality and integrity through rigorous testing and validation. Perform data extraction from SAP or ERP systems when necessary. Performance Optimization: Monitor and optimize the performance of data pipelines and ETL processes. Implement best practices for data management, including data governance, security, and compliance. Collaboration and Communication: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions. Collaborate with cross-functional teams to design and implement data solutions that meet business needs. Documentation and Maintenance: Document technical solutions, processes, and workflows. Maintain and troubleshoot existing ETL pipelines and data integrations. Qualifications: Education: Bachelor's degree in Computer Science, Information Technology, or a related field. Advanced degrees are a plus. Experience: 7+ years of experience as a Data Engineer or in a similar role. Proven experience with cloud platforms: AWS, Azure, and GCP. Hands-on experience with cloud-native ETL tools such as AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, etc. Experience with other ETL tools like Informatica, SAP Data Intelligence, etc. Experience in building and managing data lakes and data warehouses. Proficiency with data platforms like Redshift, Snowflake, BigQuery, Databricks, and Azure Synapse. Experience with data extraction from SAP or ERP systems is a plus. Strong experience with Spark and Scala for data processing. Skills: Strong programming skills in Python, Java, or Scala. Proficient in SQL and query optimization techniques. Familiarity with data modelling, ETL/ELT processes, and data warehousing concepts. Knowledge of data governance, security, and compliance best practices. Excellent problem-solving and analytical skills. Strong communication and collaboration skills. Preferred Qualifications: Experience with other data tools and technologies such as Apache Spark, or Hadoop. Certifications in cloud platforms (AWS Certified Data Analytics - Specialty, Google Professional Data Engineer, Microsoft Certified: Azure Data Engineer Associate). Experience with CI/CD pipelines and DevOps practices for data engineering What we offer in return: Remote Working: Lemongrass always has been and always will offer 100% remote work Flexibility: Work where and when you like most of the time Training: A subscription to A Cloud Guru and generous budget for taking certifications and other resources you'll find helpful State of the art tech : An opportunity to learn and run the latest industry standard tools Team: Colleagues who will challenge you giving the chance to learn from them and them from you Lemongrass Consulting is an Equal Opportunity/Affirmative Action employer. All qualified candidates will receive consideration for employment without regard to disability, protected veteran status, race, color, religious creed, national origin, citizenship, marital status, sex, sexual orientation/gender identity, age, or genetic information. Selected applicant will be subject to a background investigation.
23/12/2024
Full time
Title- Cloud Data Engineer Location- Remote Lemongrass Consulting is the leading professional and managed service provider of SAP enterprise applications running on AWS hyperscale cloud infrastructure. Our objective is to delight our customers every day by reducing the cost and increasing the agility of their SAP systems. We do this with our continuous innovation, automation, migration and operation, delivered on the world's most comprehensive cloud platforms. Our team is what makes Lemongrass exceptional and why we have the excellent reputation in the market that we enjoy today. At Lemongrass, you will work with the smartest and most motivated people in the business. We take pride in our culture of innovation and collaboration that drives us to deliver exceptional benefits to our clients every day. About the Role: We are seeking an experienced Cloud Data Engineer with a strong background in AWS, Azure, and GCP. The ideal candidate will have extensive experience with cloud-native ETL tools such as AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, and other ETL tools like Informatica, SAP Data Intelligence, etc. You will be responsible for designing, implementing, and maintaining robust data pipelines and building scalable data lakes. Experience with various data platforms like Redshift, Snowflake, Databricks, Synapse, Snowflake and others is essential. Familiarity with data extraction from SAP or ERP systems is a plus. Key Responsibilities: Design and Development: Design, develop, and maintain scalable ETL pipelines using cloud-native tools (AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, etc.). Architect and implement data lakes and data warehouses on cloud platforms (AWS, Azure, GCP). Develop and optimize data ingestion, transformation, and loading processes using Databricks, Snowflake, Redshift, BigQuery and Azure Synapse. Implement ETL processes using tools like Informatica, SAP Data Intelligence, and others. Develop and optimize data processing jobs using Spark Scala. Data Integration and Management: Integrate various data sources, including relational databases, APIs, unstructured data, and ERP systems into the data lake. Ensure data quality and integrity through rigorous testing and validation. Perform data extraction from SAP or ERP systems when necessary. Performance Optimization: Monitor and optimize the performance of data pipelines and ETL processes. Implement best practices for data management, including data governance, security, and compliance. Collaboration and Communication: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions. Collaborate with cross-functional teams to design and implement data solutions that meet business needs. Documentation and Maintenance: Document technical solutions, processes, and workflows. Maintain and troubleshoot existing ETL pipelines and data integrations. Qualifications: Education: Bachelor's degree in Computer Science, Information Technology, or a related field. Advanced degrees are a plus. Experience: 7+ years of experience as a Data Engineer or in a similar role. Proven experience with cloud platforms: AWS, Azure, and GCP. Hands-on experience with cloud-native ETL tools such as AWS DMS, AWS Glue, Kafka, Azure Data Factory, GCP Dataflow, etc. Experience with other ETL tools like Informatica, SAP Data Intelligence, etc. Experience in building and managing data lakes and data warehouses. Proficiency with data platforms like Redshift, Snowflake, BigQuery, Databricks, and Azure Synapse. Experience with data extraction from SAP or ERP systems is a plus. Strong experience with Spark and Scala for data processing. Skills: Strong programming skills in Python, Java, or Scala. Proficient in SQL and query optimization techniques. Familiarity with data modelling, ETL/ELT processes, and data warehousing concepts. Knowledge of data governance, security, and compliance best practices. Excellent problem-solving and analytical skills. Strong communication and collaboration skills. Preferred Qualifications: Experience with other data tools and technologies such as Apache Spark, or Hadoop. Certifications in cloud platforms (AWS Certified Data Analytics - Specialty, Google Professional Data Engineer, Microsoft Certified: Azure Data Engineer Associate). Experience with CI/CD pipelines and DevOps practices for data engineering What we offer in return: Remote Working: Lemongrass always has been and always will offer 100% remote work Flexibility: Work where and when you like most of the time Training: A subscription to A Cloud Guru and generous budget for taking certifications and other resources you'll find helpful State of the art tech : An opportunity to learn and run the latest industry standard tools Team: Colleagues who will challenge you giving the chance to learn from them and them from you Lemongrass Consulting is an Equal Opportunity/Affirmative Action employer. All qualified candidates will receive consideration for employment without regard to disability, protected veteran status, race, color, religious creed, national origin, citizenship, marital status, sex, sexual orientation/gender identity, age, or genetic information. Selected applicant will be subject to a background investigation.
My consumer electronics distributor is seeking a skilled developer with strong data warehouse and ETL processing, Microsoft SQL and Azure Data Platform experience and data development environment with Microsoft SQL and Azure Data Platform including Power BI service and Machine learning knowledge. The position is fully remote and is Outside IR35 for all UK-based candidates, offering a 6-month contract. All EU candidates are welcome. Able to transform conceptual designs into reliable and scalable processes by design, development, and maintenance of Cloud Database services. Able to transform conceptual design and implement end to end builds into the Azure SQL Database. Collaborate with team to identify and document analytic needs and requirements and works with team to design, develop, test, implement and support DW build outs. Will collaborate with team members to ensure functional and technical solutions and look for innovative and next generation solutions for solving data challenges. Will play an active role in quality assurance for data and solution initiatives by attending team meetings and providing necessary expertise for BI team. Technical Skills: Strong and SQL Server management system, Azure Data Platform and Power BI experience Strong understanding of relational and dimensional modelling, with experience using various tooling (Azure, On-Premises, Power BI) Should have experience in cloud migration and integration Ability to manage deployments and take on technical leadership responsibilities of database projects Create data transformations and build simple to complex data pipelines & data flows Strong experience in Azure ecosystem such as ETL Jobs, Azure Data Factory (ADF), Azure Synapse Analytics, Azure Data Pipeline, Data Flows & Data Bricks, Azure SQL Database Knowledge of other Azure Services like Azure Functions, Logical Apps, APIs or ETL Technologies Strong experience in SQL writing, T-SQL tuning/optimization, and analytics Microsoft Power BI is associated with reports, datasets, dashboards, KPI, visualizations, apps, workspaces and security Could provide advice on Power BI architecture for size, speed and functionality of reports Experience in connecting to data sources, importing data and transforming data Perform data manipulation and cleansing to prepare datasets for visualization Proficient with developing Power BI dataset and structures, based on data from SQL Server and other data sources, Data Lake, API calls Proficient with DAX code including aggregations, calculations, classifications, and hierarchies Recommend additional options for reporting and analytics and continuously improve performance Design effective layouts using Data Visualizations best practices Deploy Power BI scripts, perform detailed analytics and facilitate the needs analysis through implementation Experience with automation using a Scripting language (Python, Perl) Functions as a primary contributor in designing, coding, testing, debugging, documenting and supporting established specifications and business requirements to deliver business value Ability to perform and lead code reviews and re-enforce standards and consistency Performs problem resolutions and root cause analysis along with recommending and implementing preventive techniques If you're interested, please forward your CV to (see below) All recommendations are welcome. Lawrence Harvey is acting as an Employment Business in regards to this position.
06/11/2021
Contractor
My consumer electronics distributor is seeking a skilled developer with strong data warehouse and ETL processing, Microsoft SQL and Azure Data Platform experience and data development environment with Microsoft SQL and Azure Data Platform including Power BI service and Machine learning knowledge. The position is fully remote and is Outside IR35 for all UK-based candidates, offering a 6-month contract. All EU candidates are welcome. Able to transform conceptual designs into reliable and scalable processes by design, development, and maintenance of Cloud Database services. Able to transform conceptual design and implement end to end builds into the Azure SQL Database. Collaborate with team to identify and document analytic needs and requirements and works with team to design, develop, test, implement and support DW build outs. Will collaborate with team members to ensure functional and technical solutions and look for innovative and next generation solutions for solving data challenges. Will play an active role in quality assurance for data and solution initiatives by attending team meetings and providing necessary expertise for BI team. Technical Skills: Strong and SQL Server management system, Azure Data Platform and Power BI experience Strong understanding of relational and dimensional modelling, with experience using various tooling (Azure, On-Premises, Power BI) Should have experience in cloud migration and integration Ability to manage deployments and take on technical leadership responsibilities of database projects Create data transformations and build simple to complex data pipelines & data flows Strong experience in Azure ecosystem such as ETL Jobs, Azure Data Factory (ADF), Azure Synapse Analytics, Azure Data Pipeline, Data Flows & Data Bricks, Azure SQL Database Knowledge of other Azure Services like Azure Functions, Logical Apps, APIs or ETL Technologies Strong experience in SQL writing, T-SQL tuning/optimization, and analytics Microsoft Power BI is associated with reports, datasets, dashboards, KPI, visualizations, apps, workspaces and security Could provide advice on Power BI architecture for size, speed and functionality of reports Experience in connecting to data sources, importing data and transforming data Perform data manipulation and cleansing to prepare datasets for visualization Proficient with developing Power BI dataset and structures, based on data from SQL Server and other data sources, Data Lake, API calls Proficient with DAX code including aggregations, calculations, classifications, and hierarchies Recommend additional options for reporting and analytics and continuously improve performance Design effective layouts using Data Visualizations best practices Deploy Power BI scripts, perform detailed analytics and facilitate the needs analysis through implementation Experience with automation using a Scripting language (Python, Perl) Functions as a primary contributor in designing, coding, testing, debugging, documenting and supporting established specifications and business requirements to deliver business value Ability to perform and lead code reviews and re-enforce standards and consistency Performs problem resolutions and root cause analysis along with recommending and implementing preventive techniques If you're interested, please forward your CV to (see below) All recommendations are welcome. Lawrence Harvey is acting as an Employment Business in regards to this position.
