Job Purpose
This Senior Data Engineer will play a key role in the evolution of our data platform, driving innovation and delivering high-quality data solutions. This role will focus on both migration from our current Data Warehouse to an AWS ecosystem, and developing cutting edge real-time data solutions including automation, business intelligence, and enabling analytics.
Job Duties
- Design, develop, and maintain data pipelines and ETL processes using AWS services such as AWS Glue, AWS Lambda and AWS S3.
- Support the migration of the existing Data Warehouse from SQL Server to AWS through S3 and Redshift.
- Develop and implement data quality checks and validation procedures.
- Design and implement best practice data lakehouse architectures and data warehousing solutions.
- Collaborate with data scientists and analysts to support the deployment of machine learning and advanced analytical solutions.
- Develop and maintain data documentation and operational procedures.
- Investigate and resolve data quality issues and performance bottlenecks.
- Stay abreast of the latest data technologies and industry best practices.
- Mentor junior data engineers and provide technical guidance to other team members where applicable.
- Contribute to the development and improvement of data platform best practices.
Knowledge, Skills and Experience
- Good understanding of AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR.
- Proficiency in Python, SQL, Pipeline Orchestration, and data warehousing concepts. Ability to diagnose and resolve complex data issues.
- Ability to effectively communicate with both technical and non-technical stakeholders.
- Experience with data lakehouse architectures and data warehousing solutions.
- Experience in migrating data from on-premises systems to cloud environments would be desirable.
- Experience with Agile development methodologies.
- Experience with data security and privacy best practices.
What is in it for me?
- A competitive rate of pay and pension contribution (£55,000 - £80,000)
- Generous discretionary bonus schemes, incentives and competitions
- An annual leave entitlement that increases with length of service
- Access to an online GP 24/7, 365 days a year for you and your immediate family.
- Employee wellbeing support through our Employee Assistance Programme
- Enhanced Maternity & Paternity Pay
- Long Service Recognition
- Access to a pay day savings scheme, financial coach and up to 40% of your earned wage ahead of payday, through Wagestream.