Data Engineer

This is an exciting opportunity to join a dynamic data migration consultancy based in Watford, Hertfordshire, currently operating a hybrid flexible working model in accordance with government guidelines. We are looking for permanent experienced Data Engineers who will be critical to this fast-growing company's success in delivering large scale data integrations and migrations in the cloud.

Applicants must have a keen interest in emerging technologies, and a willingness to learn new skills. We offer investment in training and the chance to work in a fast-paced company, with on-premise and cloud environments. This is a fantastic opportunity for you to expand your skills in both data migration and build of data streaming/integration systems.

This role is a permanent role. Applicants that are currently working as contractors must clearly state that they now seek a permanent position.

Initially you will be part of our strong delivery team delivering a critical national infrastructure data service that integrates a Legacy on-premise system to a new AWS system working with our team of data engineers, architects and leads.

On this project you will be expected to:

• Implement data flows to connect operational systems and provide data for analytics systems
• Document source-to-target mappings
• Assist product development teams in consuming the data streams
• Undertake reconciliation of the integrated systems
• Engineer data flows to enable scaling and repeatable use
• Support the build of data streaming systems
• Write ETL scripts and code to make sure the ETL process performs optimally

Essential skills:

• SQL and No SQL Databases (3 years minimum)
• Experience and knowledge of using JSON
• Programming and build. You can design, code, test, correct and document programs and scripts.
  • SQL Development background (3 years minimum)
  • Good working knowledge of Kafka
  • Good SQL skills, able to write queries and stored procedures
• Good communication skills, you will be able to translate technical concepts into non-technical language and understand what communication is required for internal and external stakeholders
• Experience of undertaking data profiling and source system analysis, and presenting clear insights to colleagues to support the end use of the data
• An understanding of the concepts and principles of Datamodelling and the ability to produce and maintain data models
• Experience of developing ETL or integration solutions

Desirable skills:

• Experience of Public Sector
• Elastic Search/Kibana
• Current SC Clearance
• Exposure to AWS
• RDS
• DynamoDB

Security Clearance is required to SC level so you must be willing to undergo clearance as part of the recruitment process. Five years' residency in the UK is a mandatory requirement of security clearance, as is ownership of a passport.