Data Engineer

Data Engineer

Location: London

Length: 12 Months

Rate: £57.00 - £63.00 per hour (40 hours p/w)

Hours: 9am-6pm

Our client is looking for a Data Engineer to not only build data pipelines but also extend the next generation of their data tools.

As a Data Engineer, you will develop a clear sense of connection with our client's organization and leadership - as Data Engineering is the eyes through which they see the product.

As a member of Infrastructure Strategy Data Engineering, you will belong to a centralized Data Science/Data Engineering team who partners closely with teams in the company Infrastructure organization. Through the consulting-nature of our client's team, you will contribute to a variety of projects and technologies, depending on partner needs. Projects include analytics, ML modeling, tooling, services, and more. Data Engineer, Infrastructure Strategy

Responsibilities

• Partner with leadership, engineers, program managers and data scientists to understand data needs.
• Design, build and launch extremely efficient and reliable data pipelines to move data across a number of platforms including Data Warehouse, online caches and real-time systems.
• Communicate, at scale, through multiple mediums: Presentations, dashboards, company-wide datasets, bots and more.
• Educate your partners: Use your data and analytics experience to 'see what's missing', identifying and addressing gaps in their existing logging and processes.
• Broad range of partners equates to a broad range of projects and deliverables: ML Models, datasets, measurements, services, tools and process.
• Leverage data and business principles to solve large scale web, mobile and data infrastructure problems.
• Build data expertise and own data quality for your areas.

Minimum Qualifications

• Strong SQL skills are a MUST Data Modeling skills are a MUST and considered a portion of the SQL signals they look for For Coding, should have familiarity with major concepts and a comfort level with at least one language of the candidate's choice.
• Problem Solving is required Partnership is less of a thing here as most of the work will be directed by FTE Data Engineers, however this is a PLUS if the candidate exhibits good soft skills 5+ years of Python development experience.
• 5+ years of SQL experience. 3+ years of experience with workflow management engines (i.e. Airflow, Luigi, Prefect, Dagster, digdag.io, Google Cloud Composer, AWS Step Functions, Azure Data Factory, UC4, Control-M).
• 3+ years experience with Data Modeling. Experience analyzing data to discover opportunities and address gaps.
• 5+ years experience in custom ETL design, implementation and maintenance. Experience working with cloud or on-prem Big Data/MPP analytics platform(i.e. Netezza, Teradata, AWS Redshift, Google BigQuery, Azure Data Warehouse, or similar).

Preferred Qualifications

• Experience with more than one coding language.
• Designing and implementing real-time pipelines.
• Experience with data quality and validation.
• Experience with SQL performance tuning and e2e process optimization.
• Experience with anomaly/outlier detection.
• Experience with notebook-based Data Science workflow.
• Experience with Airflow. Experience querying massive datasets using Spark, Presto, Hive, Impala, etc.