Scope AT Limited
04/11/2021
Full time
Big Data Engineer - Pyspark/Spark/API/Hadoop/Python/SQL/UNIX Top Consultancy is looking for a Big Data Engineer to join them in a fully remote permanent position. You will be working with a major Investment Bank. Requirements: Dataframe API PySpark Understanding partitioning very well Analysing and performance tuning Spark queries eg looking at the DAG Knowledge of Hadoop and its ecosystem of technologies especially Hive OOP concepts using Python Understand Data Structures & their Implementation Contributes to quality assurance by best practices and effective testing. Ensures development happens for all Software Components in accordance with Detailed Software Requirements specification, the functional design and the technical design document. Basic knowledge of UNIX Ability to analyse databases directly using query language tools such as SQL Experience on ETL process on Big Data Have deep understanding of data relationships, normalisation Contributes to problem solving and root cause analysis. By applying to this job you are sending us your CV, so we are keen for you to see our Privacy Notice. Scope AT acts as an employment agency for Permanent Recruitment and an employment business for the supply of temporary workers. By applying for this job you accept the Terms and Conditions, Data Protection Policy, Privacy Notice and Disclaimers which can be found on the web