Data Technical Lead Location: Hybrid - London Package: Negotiable + Benefits As a Technical Lead for our Data Team, you will play a critical role in shaping our data strategy, driving technical excellence, and ensuring the successful delivery of data services solutions. You will provide technical and solution thought leadership to a squad of data engineers to ensure we develop best in class data services solutions that enable us to achieve our business objectives. Additionally you will be an active participant and contributor in the wider Brown and Brown Data & Analytics Community of Practice The day to day: You will be an experienced technical leader with deep technical skills in data engineering. Your primary responsibilities will include: Planning Data Projects : As a data tech lead, you will work closely with the delivery lead strategising and planning data projects. This involves defining project scope, objectives, timelines, and resource allocation. You'll collaborate with stakeholders to ensure alignment with business goals. Building Data Systems : Your expertise lies in designing and implementing data systems. This includes creating data pipelines, producing visualisation and business report, and everything in between. You will be the most technically proficient member of the squad. Technical Decision-Making: As a data tech lead, you'll make critical technical decisions related to data architecture, tool selection, and data processing. Your decisions will be made in the context of our Group data strategy, ensuring that choices are operable and in alignment with the data architecture for Europe. Your choices impact the overall success of data projects. Problem solving: You will work with squad members to identify and fix technical problems. Data Governance and Quality Assurance : Ensuring data governance and quality is paramount. You will influence and champion corporate data policy and standards and ensure their implementation across Brown and Brown Europe. Hiring : You will be a key actor in the hiring process of teammates into the data tribe. Training and Enabling : You will be part of a team of professionals with diverse skills. We are looking to build elite capabilities, your role involves mentoring, coaching, and ensuring the squads technical growth. Effective communication and collaboration are essential for team success. Best Practice - You will keep current with emerging data trends and practices and bring those into the data team. You will act as a thought leader and be an active part of the wider Brown & Brown Data & Analytics Community of Practice About you: Experience as a Tech Lead / Senior developer in data engineering projects Experience within technology software / data engineering Experience working with multi-TB scale data solutions Experience implementing cloud-based data and analytics solutions Experience implementing and operating within robust data governance principles - metadata, discovery, tagging and classification, quality, catalogues, inventory and data lineage Technical Skills - Deep hands on experience with the following data technologies: Microsoft stack including, Azure Data factory, Azure Data Lake Storage, Azure SQL, Azure Cosmos DB, Azure Power Platform, primarily power BI DataBricks Denodo Datawarehouse, Data Lake & Data Mesh implementation The rewards: A negotiable basic salary and all the normal benefits you'd expect (Holiday, company pension etc.) A collaborative, open and honest environment that is designed to deliver the best outcomes to our clients and staff A flexible working methodology to enable you to be where you need to be, if you don't need to be in an office then don't, if you want to be in an office your welcome to use one. An environment built around supporting and developing our staff with funding available for relevant professional qualifications.
Apr 25, 2024
Full time
Data Technical Lead Location: Hybrid - London Package: Negotiable + Benefits As a Technical Lead for our Data Team, you will play a critical role in shaping our data strategy, driving technical excellence, and ensuring the successful delivery of data services solutions. You will provide technical and solution thought leadership to a squad of data engineers to ensure we develop best in class data services solutions that enable us to achieve our business objectives. Additionally you will be an active participant and contributor in the wider Brown and Brown Data & Analytics Community of Practice The day to day: You will be an experienced technical leader with deep technical skills in data engineering. Your primary responsibilities will include: Planning Data Projects : As a data tech lead, you will work closely with the delivery lead strategising and planning data projects. This involves defining project scope, objectives, timelines, and resource allocation. You'll collaborate with stakeholders to ensure alignment with business goals. Building Data Systems : Your expertise lies in designing and implementing data systems. This includes creating data pipelines, producing visualisation and business report, and everything in between. You will be the most technically proficient member of the squad. Technical Decision-Making: As a data tech lead, you'll make critical technical decisions related to data architecture, tool selection, and data processing. Your decisions will be made in the context of our Group data strategy, ensuring that choices are operable and in alignment with the data architecture for Europe. Your choices impact the overall success of data projects. Problem solving: You will work with squad members to identify and fix technical problems. Data Governance and Quality Assurance : Ensuring data governance and quality is paramount. You will influence and champion corporate data policy and standards and ensure their implementation across Brown and Brown Europe. Hiring : You will be a key actor in the hiring process of teammates into the data tribe. Training and Enabling : You will be part of a team of professionals with diverse skills. We are looking to build elite capabilities, your role involves mentoring, coaching, and ensuring the squads technical growth. Effective communication and collaboration are essential for team success. Best Practice - You will keep current with emerging data trends and practices and bring those into the data team. You will act as a thought leader and be an active part of the wider Brown & Brown Data & Analytics Community of Practice About you: Experience as a Tech Lead / Senior developer in data engineering projects Experience within technology software / data engineering Experience working with multi-TB scale data solutions Experience implementing cloud-based data and analytics solutions Experience implementing and operating within robust data governance principles - metadata, discovery, tagging and classification, quality, catalogues, inventory and data lineage Technical Skills - Deep hands on experience with the following data technologies: Microsoft stack including, Azure Data factory, Azure Data Lake Storage, Azure SQL, Azure Cosmos DB, Azure Power Platform, primarily power BI DataBricks Denodo Datawarehouse, Data Lake & Data Mesh implementation The rewards: A negotiable basic salary and all the normal benefits you'd expect (Holiday, company pension etc.) A collaborative, open and honest environment that is designed to deliver the best outcomes to our clients and staff A flexible working methodology to enable you to be where you need to be, if you don't need to be in an office then don't, if you want to be in an office your welcome to use one. An environment built around supporting and developing our staff with funding available for relevant professional qualifications.
I'm recruiting for a Data Engineer for a global agile tech consultancy - based in Sheffield + hybrid flexible working model, Inside IR35, £400-500.00 a day. Senior Data Engineer Qualifications: Must-have skills: SQL, Azure Databricks, Azure Data Lake, Azure Data Factory Experience with domain and Datamodelling Nice to have skills: Power BI, Spark Developing data processing pipelines Providing testing and assurance for data pipelines to ensure transformations are correct Scripting experience eg Python, Powershell Incorporate assets required for data pipelines into CI/CD pipelines Required start date ASAP, 6-month contract with a high potential to be extended If you feel you are suitable please apply for the role Xpertise Recruitment acts as an employment agency
Nov 05, 2021
Contractor
I'm recruiting for a Data Engineer for a global agile tech consultancy - based in Sheffield + hybrid flexible working model, Inside IR35, £400-500.00 a day. Senior Data Engineer Qualifications: Must-have skills: SQL, Azure Databricks, Azure Data Lake, Azure Data Factory Experience with domain and Datamodelling Nice to have skills: Power BI, Spark Developing data processing pipelines Providing testing and assurance for data pipelines to ensure transformations are correct Scripting experience eg Python, Powershell Incorporate assets required for data pipelines into CI/CD pipelines Required start date ASAP, 6-month contract with a high potential to be extended If you feel you are suitable please apply for the role Xpertise Recruitment acts as an employment agency
Team Overview Data is central to Ofcom's work. We use data from a wide range of sources to understand the dynamics of the sectors we regulate and to provide key insights to inform policy decisions. In addition to existing regulatory responsibilities, there will be exciting opportunities to contribute to Ofcom's new duties in relation to the regulation of Online Harms. It is expected that this new area of work will expand the scope of data-driven activities at Ofcom in terms of the variety and volume of data, as well as the range of analysis we do. To ensure that Ofcom has the appropriate data capabilities to undertake the analysis effectively, ICT has created a new role for an Data Solutions Architect Purpose of the Role The Data Solutions Architect works closely with ICT and the Data Innovation Hub. You will architect, design, implement and oversee the operations of data solutions that empower data professionals to efficiently and effectively deliver their work. Candidates will exhibit critical thinking skills, the ability to synthesize complex problems, and have relevant skills and experience for enabling the transformation of data to create solutions that add value to a myriad of business requirements. You must have a deep understanding of the full data lifecycle and the role that high-quality data plays across applications, machine learning, business analytics, and reporting. You will lead the design and development of solution architectures in response to business requirements. This includes identifying and evaluating alternative architectures, their trade-offs in cost, performance and scalability. And ensuring that the relevant technical strategies, policies, standards and practices (including security) are applied correctly. The end to end solution will be fit for purpose - i.e. meet the needs of business, the agreed requirements, and support the strategic direction of Ofcom. Work with IT teams, business analysts and data analytics teams to understand data consumers' needs and develop solutions By maintaining your knowledge of emerging trends in data usage, tools and analysis techniques you will support our on-going development activities and continually promote data innovation as a means to achieve the business outcomes for specific Groups, and Ofcom. You will need to be self-motivated, an effective communicator and have a collaborative delivery approach.You will work in a collaborative cross-functional environment and interact with the full spectrum of colleagues (data engineers, data analysts, data scientists, operational support and policy makers), and you will need to inform and influence senior managers. Requirements of the Role Build strong relationships with colleagues across the business, understanding their motivations behind projects and own technical activities to translate business requirements (both functional and non-functional) into a solution. Ensuring the required business value is delivered. Fostering a customer centric approach to ensure delivery of business value and an iterative approach that responds to feedback and changing needs. Perform deep dives into technical areas to solve a specific solution or design challenges. Using trials or POCs to prove or discount an approach, to critic your own design. You will be responsible for ensuring that the solutions you help deliver form an integral part of the ICT estate and align with the wider reference architecture and domain roadmaps. Manage stakeholder expectations and be flexible, working on many different projects and topics at the same time period. Manage proactive and reactive communication. Facilitate difficult discussions within the team or with diverse senior stakeholders and external / 3rd parties as necessary. Provide documentation of solutions detailing the business, data, application and technology layers. Work with Data Engineers to define data pipelines and data lakes, covering the ingression, ETL or ELT, and the cataloguing of data. Takes overall responsibility for planning effective data storage, cost, security, quality, sharing, availability, retention and publishing within the organisation. Develops analytics policy, standards and guidelines Ensuring successful transitions for solutions into production ensuring production support have the necessary knowledge and documentation to support the service. Skills, knowledge and experience Robust Data and Technical/Solutions Architecture skills - sets direction for and possesses a deep understanding of architecture and strategies which integrate with industry trends e.g. TOGAF Hands-on experience with analytical tools and languages such as: Python, R, SQL, Azure Data Factory (and SSIS), Databricks, Power BI, Git etc. Experience of infrastructure: cloud-based technologies used for storage, data lakes, data warehouses, data streaming, Databases, ETL / ELT / Transformation. Experience of DevOps/ DataOps methods in development of data solutions to ensure pipelines and processes can be automated and controlled. Experience with Cloud based data and analytical initiatives utilising Azure, AWS, Google Cloud or similar cloud services Experience of working closely with Data Professionals (E.g. Data Scientists and Data Analysts) to understanding their needs. Experience of implementating of statistical, Artificial Intelligence, Machine Learning and Deep learning applications. Experience with integrations (e.g. via APIs) with external vendors to share data between organizations Experience of working with external technology supplier and service providers to deliver business solutions SFIA Skill Enterprise and business architecture STPL - Level 5 Solution architecture ARCH - Level 5 Requirements definition and management REQM- Level 5 Database design DBDS- Level 5 Analytics INAN- Level 4 Emerging Technology Monitoring (EMRG)- Level 4 Relationship Management RLMT- Level 5
Nov 04, 2021
Full time
Team Overview Data is central to Ofcom's work. We use data from a wide range of sources to understand the dynamics of the sectors we regulate and to provide key insights to inform policy decisions. In addition to existing regulatory responsibilities, there will be exciting opportunities to contribute to Ofcom's new duties in relation to the regulation of Online Harms. It is expected that this new area of work will expand the scope of data-driven activities at Ofcom in terms of the variety and volume of data, as well as the range of analysis we do. To ensure that Ofcom has the appropriate data capabilities to undertake the analysis effectively, ICT has created a new role for an Data Solutions Architect Purpose of the Role The Data Solutions Architect works closely with ICT and the Data Innovation Hub. You will architect, design, implement and oversee the operations of data solutions that empower data professionals to efficiently and effectively deliver their work. Candidates will exhibit critical thinking skills, the ability to synthesize complex problems, and have relevant skills and experience for enabling the transformation of data to create solutions that add value to a myriad of business requirements. You must have a deep understanding of the full data lifecycle and the role that high-quality data plays across applications, machine learning, business analytics, and reporting. You will lead the design and development of solution architectures in response to business requirements. This includes identifying and evaluating alternative architectures, their trade-offs in cost, performance and scalability. And ensuring that the relevant technical strategies, policies, standards and practices (including security) are applied correctly. The end to end solution will be fit for purpose - i.e. meet the needs of business, the agreed requirements, and support the strategic direction of Ofcom. Work with IT teams, business analysts and data analytics teams to understand data consumers' needs and develop solutions By maintaining your knowledge of emerging trends in data usage, tools and analysis techniques you will support our on-going development activities and continually promote data innovation as a means to achieve the business outcomes for specific Groups, and Ofcom. You will need to be self-motivated, an effective communicator and have a collaborative delivery approach.You will work in a collaborative cross-functional environment and interact with the full spectrum of colleagues (data engineers, data analysts, data scientists, operational support and policy makers), and you will need to inform and influence senior managers. Requirements of the Role Build strong relationships with colleagues across the business, understanding their motivations behind projects and own technical activities to translate business requirements (both functional and non-functional) into a solution. Ensuring the required business value is delivered. Fostering a customer centric approach to ensure delivery of business value and an iterative approach that responds to feedback and changing needs. Perform deep dives into technical areas to solve a specific solution or design challenges. Using trials or POCs to prove or discount an approach, to critic your own design. You will be responsible for ensuring that the solutions you help deliver form an integral part of the ICT estate and align with the wider reference architecture and domain roadmaps. Manage stakeholder expectations and be flexible, working on many different projects and topics at the same time period. Manage proactive and reactive communication. Facilitate difficult discussions within the team or with diverse senior stakeholders and external / 3rd parties as necessary. Provide documentation of solutions detailing the business, data, application and technology layers. Work with Data Engineers to define data pipelines and data lakes, covering the ingression, ETL or ELT, and the cataloguing of data. Takes overall responsibility for planning effective data storage, cost, security, quality, sharing, availability, retention and publishing within the organisation. Develops analytics policy, standards and guidelines Ensuring successful transitions for solutions into production ensuring production support have the necessary knowledge and documentation to support the service. Skills, knowledge and experience Robust Data and Technical/Solutions Architecture skills - sets direction for and possesses a deep understanding of architecture and strategies which integrate with industry trends e.g. TOGAF Hands-on experience with analytical tools and languages such as: Python, R, SQL, Azure Data Factory (and SSIS), Databricks, Power BI, Git etc. Experience of infrastructure: cloud-based technologies used for storage, data lakes, data warehouses, data streaming, Databases, ETL / ELT / Transformation. Experience of DevOps/ DataOps methods in development of data solutions to ensure pipelines and processes can be automated and controlled. Experience with Cloud based data and analytical initiatives utilising Azure, AWS, Google Cloud or similar cloud services Experience of working closely with Data Professionals (E.g. Data Scientists and Data Analysts) to understanding their needs. Experience of implementating of statistical, Artificial Intelligence, Machine Learning and Deep learning applications. Experience with integrations (e.g. via APIs) with external vendors to share data between organizations Experience of working with external technology supplier and service providers to deliver business solutions SFIA Skill Enterprise and business architecture STPL - Level 5 Solution architecture ARCH - Level 5 Requirements definition and management REQM- Level 5 Database design DBDS- Level 5 Analytics INAN- Level 4 Emerging Technology Monitoring (EMRG)- Level 4 Relationship Management RLMT- Level 5
Team Overview Data is central to Ofcom's work. We use data from a wide range of sources to understand the dynamics of the sectors we regulate and to provide key insights to inform policy decisions. In addition to existing regulatory responsibilities, there will be exciting opportunities to contribute to Ofcom's new duties in relation to the regulation of Online Harms. It is expected that this new area of work will expand the scope of data-driven activities at Ofcom in terms of the variety and volume of data, as well as the range of analysis we do. To ensure that Ofcom has the appropriate data capabilities to undertake the analysis effectively, ICT has created a new role for an Data Solutions Architect Purpose of the Role The Data Solutions Architect works closely with ICT and the Data Innovation Hub. You will architect, design, implement and oversee the operations of data solutions that empower data professionals to efficiently and effectively deliver their work. Candidates will exhibit critical thinking skills, the ability to synthesize complex problems, and have relevant skills and experience for enabling the transformation of data to create solutions that add value to a myriad of business requirements. You must have a deep understanding of the full data lifecycle and the role that high-quality data plays across applications, machine learning, business analytics, and reporting. You will lead the design and development of solution architectures in response to business requirements. This includes identifying and evaluating alternative architectures, their trade-offs in cost, performance and scalability. And ensuring that the relevant technical strategies, policies, standards and practices (including security) are applied correctly. The end to end solution will be fit for purpose - i.e. meet the needs of business, the agreed requirements, and support the strategic direction of Ofcom. Work with IT teams, business analysts and data analytics teams to understand data consumers' needs and develop solutions By maintaining your knowledge of emerging trends in data usage, tools and analysis techniques you will support our on-going development activities and continually promote data innovation as a means to achieve the business outcomes for specific Groups, and Ofcom. You will need to be self-motivated, an effective communicator and have a collaborative delivery approach.You will work in a collaborative cross-functional environment and interact with the full spectrum of colleagues (data engineers, data analysts, data scientists, operational support and policy makers), and you will need to inform and influence senior managers. Requirements of the Role Build strong relationships with colleagues across the business, understanding their motivations behind projects and own technical activities to translate business requirements (both functional and non-functional) into a solution. Ensuring the required business value is delivered. Fostering a customer centric approach to ensure delivery of business value and an iterative approach that responds to feedback and changing needs. Perform deep dives into technical areas to solve a specific solution or design challenges. Using trials or POCs to prove or discount an approach, to critic your own design. You will be responsible for ensuring that the solutions you help deliver form an integral part of the ICT estate and align with the wider reference architecture and domain roadmaps. Manage stakeholder expectations and be flexible, working on many different projects and topics at the same time period. Manage proactive and reactive communication. Facilitate difficult discussions within the team or with diverse senior stakeholders and external / 3rd parties as necessary. Provide documentation of solutions detailing the business, data, application and technology layers. Work with Data Engineers to define data pipelines and data lakes, covering the ingression, ETL or ELT, and the cataloguing of data. Takes overall responsibility for planning effective data storage, cost, security, quality, sharing, availability, retention and publishing within the organisation. Develops analytics policy, standards and guidelines Ensuring successful transitions for solutions into production ensuring production support have the necessary knowledge and documentation to support the service. Skills, knowledge and experience Robust Data and Technical/Solutions Architecture skills - sets direction for and possesses a deep understanding of architecture and strategies which integrate with industry trends e.g. TOGAF Hands-on experience with analytical tools and languages such as: Python, R, SQL, Azure Data Factory (and SSIS), Databricks, Power BI, Git etc. Experience of infrastructure: cloud-based technologies used for storage, data lakes, data warehouses, data streaming, Databases, ETL / ELT / Transformation. Experience of DevOps/ DataOps methods in development of data solutions to ensure pipelines and processes can be automated and controlled. Experience with Cloud based data and analytical initiatives utilising Azure, AWS, Google Cloud or similar cloud services Experience of working closely with Data Professionals (E.g. Data Scientists and Data Analysts) to understanding their needs. Experience of implementating of statistical, Artificial Intelligence, Machine Learning and Deep learning applications. Experience with integrations (e.g. via APIs) with external vendors to share data between organizations Experience of working with external technology supplier and service providers to deliver business solutions SFIA Skill Enterprise and business architecture STPL - Level 5 Solution architecture ARCH - Level 5 Requirements definition and management REQM- Level 5 Database design DBDS- Level 5 Analytics INAN- Level 4 Emerging Technology Monitoring (EMRG)- Level 4 Relationship Management RLMT- Level 5
Nov 04, 2021
Full time
Team Overview Data is central to Ofcom's work. We use data from a wide range of sources to understand the dynamics of the sectors we regulate and to provide key insights to inform policy decisions. In addition to existing regulatory responsibilities, there will be exciting opportunities to contribute to Ofcom's new duties in relation to the regulation of Online Harms. It is expected that this new area of work will expand the scope of data-driven activities at Ofcom in terms of the variety and volume of data, as well as the range of analysis we do. To ensure that Ofcom has the appropriate data capabilities to undertake the analysis effectively, ICT has created a new role for an Data Solutions Architect Purpose of the Role The Data Solutions Architect works closely with ICT and the Data Innovation Hub. You will architect, design, implement and oversee the operations of data solutions that empower data professionals to efficiently and effectively deliver their work. Candidates will exhibit critical thinking skills, the ability to synthesize complex problems, and have relevant skills and experience for enabling the transformation of data to create solutions that add value to a myriad of business requirements. You must have a deep understanding of the full data lifecycle and the role that high-quality data plays across applications, machine learning, business analytics, and reporting. You will lead the design and development of solution architectures in response to business requirements. This includes identifying and evaluating alternative architectures, their trade-offs in cost, performance and scalability. And ensuring that the relevant technical strategies, policies, standards and practices (including security) are applied correctly. The end to end solution will be fit for purpose - i.e. meet the needs of business, the agreed requirements, and support the strategic direction of Ofcom. Work with IT teams, business analysts and data analytics teams to understand data consumers' needs and develop solutions By maintaining your knowledge of emerging trends in data usage, tools and analysis techniques you will support our on-going development activities and continually promote data innovation as a means to achieve the business outcomes for specific Groups, and Ofcom. You will need to be self-motivated, an effective communicator and have a collaborative delivery approach.You will work in a collaborative cross-functional environment and interact with the full spectrum of colleagues (data engineers, data analysts, data scientists, operational support and policy makers), and you will need to inform and influence senior managers. Requirements of the Role Build strong relationships with colleagues across the business, understanding their motivations behind projects and own technical activities to translate business requirements (both functional and non-functional) into a solution. Ensuring the required business value is delivered. Fostering a customer centric approach to ensure delivery of business value and an iterative approach that responds to feedback and changing needs. Perform deep dives into technical areas to solve a specific solution or design challenges. Using trials or POCs to prove or discount an approach, to critic your own design. You will be responsible for ensuring that the solutions you help deliver form an integral part of the ICT estate and align with the wider reference architecture and domain roadmaps. Manage stakeholder expectations and be flexible, working on many different projects and topics at the same time period. Manage proactive and reactive communication. Facilitate difficult discussions within the team or with diverse senior stakeholders and external / 3rd parties as necessary. Provide documentation of solutions detailing the business, data, application and technology layers. Work with Data Engineers to define data pipelines and data lakes, covering the ingression, ETL or ELT, and the cataloguing of data. Takes overall responsibility for planning effective data storage, cost, security, quality, sharing, availability, retention and publishing within the organisation. Develops analytics policy, standards and guidelines Ensuring successful transitions for solutions into production ensuring production support have the necessary knowledge and documentation to support the service. Skills, knowledge and experience Robust Data and Technical/Solutions Architecture skills - sets direction for and possesses a deep understanding of architecture and strategies which integrate with industry trends e.g. TOGAF Hands-on experience with analytical tools and languages such as: Python, R, SQL, Azure Data Factory (and SSIS), Databricks, Power BI, Git etc. Experience of infrastructure: cloud-based technologies used for storage, data lakes, data warehouses, data streaming, Databases, ETL / ELT / Transformation. Experience of DevOps/ DataOps methods in development of data solutions to ensure pipelines and processes can be automated and controlled. Experience with Cloud based data and analytical initiatives utilising Azure, AWS, Google Cloud or similar cloud services Experience of working closely with Data Professionals (E.g. Data Scientists and Data Analysts) to understanding their needs. Experience of implementating of statistical, Artificial Intelligence, Machine Learning and Deep learning applications. Experience with integrations (e.g. via APIs) with external vendors to share data between organizations Experience of working with external technology supplier and service providers to deliver business solutions SFIA Skill Enterprise and business architecture STPL - Level 5 Solution architecture ARCH - Level 5 Requirements definition and management REQM- Level 5 Database design DBDS- Level 5 Analytics INAN- Level 4 Emerging Technology Monitoring (EMRG)- Level 4 Relationship Management RLMT- Level 5
Team Overview Data is central to Ofcom's work. We use data from a wide range of sources to understand the dynamics of the sectors we regulate and to provide key insights to inform policy decisions. In addition to existing regulatory responsibilities, there will be exciting opportunities to contribute to Ofcom's new duties in relation to the regulation of Online Harms. It is expected that this new area of work will expand the scope of data-driven activities at Ofcom in terms of the variety and volume of data, as well as the range of analysis we do. To ensure that Ofcom has the appropriate data capabilities to undertake the analysis effectively, ICT has created a new role for an Data Solutions Architect Purpose of the Role The Data Solutions Architect works closely with ICT and the Data Innovation Hub. You will architect, design, implement and oversee the operations of data solutions that empower data professionals to efficiently and effectively deliver their work. Candidates will exhibit critical thinking skills, the ability to synthesize complex problems, and have relevant skills and experience for enabling the transformation of data to create solutions that add value to a myriad of business requirements. You must have a deep understanding of the full data lifecycle and the role that high-quality data plays across applications, machine learning, business analytics, and reporting. You will lead the design and development of solution architectures in response to business requirements. This includes identifying and evaluating alternative architectures, their trade-offs in cost, performance and scalability. And ensuring that the relevant technical strategies, policies, standards and practices (including security) are applied correctly. The end to end solution will be fit for purpose - i.e. meet the needs of business, the agreed requirements, and support the strategic direction of Ofcom. Work with IT teams, business analysts and data analytics teams to understand data consumers' needs and develop solutions By maintaining your knowledge of emerging trends in data usage, tools and analysis techniques you will support our on-going development activities and continually promote data innovation as a means to achieve the business outcomes for specific Groups, and Ofcom. You will need to be self-motivated, an effective communicator and have a collaborative delivery approach.You will work in a collaborative cross-functional environment and interact with the full spectrum of colleagues (data engineers, data analysts, data scientists, operational support and policy makers), and you will need to inform and influence senior managers. Requirements of the Role Build strong relationships with colleagues across the business, understanding their motivations behind projects and own technical activities to translate business requirements (both functional and non-functional) into a solution. Ensuring the required business value is delivered. Fostering a customer centric approach to ensure delivery of business value and an iterative approach that responds to feedback and changing needs. Perform deep dives into technical areas to solve a specific solution or design challenges. Using trials or POCs to prove or discount an approach, to critic your own design. You will be responsible for ensuring that the solutions you help deliver form an integral part of the ICT estate and align with the wider reference architecture and domain roadmaps. Manage stakeholder expectations and be flexible, working on many different projects and topics at the same time period. Manage proactive and reactive communication. Facilitate difficult discussions within the team or with diverse senior stakeholders and external / 3rd parties as necessary. Provide documentation of solutions detailing the business, data, application and technology layers. Work with Data Engineers to define data pipelines and data lakes, covering the ingression, ETL or ELT, and the cataloguing of data. Takes overall responsibility for planning effective data storage, cost, security, quality, sharing, availability, retention and publishing within the organisation. Develops analytics policy, standards and guidelines Ensuring successful transitions for solutions into production ensuring production support have the necessary knowledge and documentation to support the service. Skills, knowledge and experience Robust Data and Technical/Solutions Architecture skills - sets direction for and possesses a deep understanding of architecture and strategies which integrate with industry trends e.g. TOGAF Hands-on experience with analytical tools and languages such as: Python, R, SQL, Azure Data Factory (and SSIS), Databricks, Power BI, Git etc. Experience of infrastructure: cloud-based technologies used for storage, data lakes, data warehouses, data streaming, Databases, ETL / ELT / Transformation. Experience of DevOps/ DataOps methods in development of data solutions to ensure pipelines and processes can be automated and controlled. Experience with Cloud based data and analytical initiatives utilising Azure, AWS, Google Cloud or similar cloud services Experience of working closely with Data Professionals (E.g. Data Scientists and Data Analysts) to understanding their needs. Experience of implementating of statistical, Artificial Intelligence, Machine Learning and Deep learning applications. Experience with integrations (e.g. via APIs) with external vendors to share data between organizations Experience of working with external technology supplier and service providers to deliver business solutions SFIA Skill Enterprise and business architecture STPL - Level 5 Solution architecture ARCH - Level 5 Requirements definition and management REQM- Level 5 Database design DBDS- Level 5 Analytics INAN- Level 4 Emerging Technology Monitoring (EMRG)- Level 4 Relationship Management RLMT- Level 5
Nov 04, 2021
Full time
Team Overview Data is central to Ofcom's work. We use data from a wide range of sources to understand the dynamics of the sectors we regulate and to provide key insights to inform policy decisions. In addition to existing regulatory responsibilities, there will be exciting opportunities to contribute to Ofcom's new duties in relation to the regulation of Online Harms. It is expected that this new area of work will expand the scope of data-driven activities at Ofcom in terms of the variety and volume of data, as well as the range of analysis we do. To ensure that Ofcom has the appropriate data capabilities to undertake the analysis effectively, ICT has created a new role for an Data Solutions Architect Purpose of the Role The Data Solutions Architect works closely with ICT and the Data Innovation Hub. You will architect, design, implement and oversee the operations of data solutions that empower data professionals to efficiently and effectively deliver their work. Candidates will exhibit critical thinking skills, the ability to synthesize complex problems, and have relevant skills and experience for enabling the transformation of data to create solutions that add value to a myriad of business requirements. You must have a deep understanding of the full data lifecycle and the role that high-quality data plays across applications, machine learning, business analytics, and reporting. You will lead the design and development of solution architectures in response to business requirements. This includes identifying and evaluating alternative architectures, their trade-offs in cost, performance and scalability. And ensuring that the relevant technical strategies, policies, standards and practices (including security) are applied correctly. The end to end solution will be fit for purpose - i.e. meet the needs of business, the agreed requirements, and support the strategic direction of Ofcom. Work with IT teams, business analysts and data analytics teams to understand data consumers' needs and develop solutions By maintaining your knowledge of emerging trends in data usage, tools and analysis techniques you will support our on-going development activities and continually promote data innovation as a means to achieve the business outcomes for specific Groups, and Ofcom. You will need to be self-motivated, an effective communicator and have a collaborative delivery approach.You will work in a collaborative cross-functional environment and interact with the full spectrum of colleagues (data engineers, data analysts, data scientists, operational support and policy makers), and you will need to inform and influence senior managers. Requirements of the Role Build strong relationships with colleagues across the business, understanding their motivations behind projects and own technical activities to translate business requirements (both functional and non-functional) into a solution. Ensuring the required business value is delivered. Fostering a customer centric approach to ensure delivery of business value and an iterative approach that responds to feedback and changing needs. Perform deep dives into technical areas to solve a specific solution or design challenges. Using trials or POCs to prove or discount an approach, to critic your own design. You will be responsible for ensuring that the solutions you help deliver form an integral part of the ICT estate and align with the wider reference architecture and domain roadmaps. Manage stakeholder expectations and be flexible, working on many different projects and topics at the same time period. Manage proactive and reactive communication. Facilitate difficult discussions within the team or with diverse senior stakeholders and external / 3rd parties as necessary. Provide documentation of solutions detailing the business, data, application and technology layers. Work with Data Engineers to define data pipelines and data lakes, covering the ingression, ETL or ELT, and the cataloguing of data. Takes overall responsibility for planning effective data storage, cost, security, quality, sharing, availability, retention and publishing within the organisation. Develops analytics policy, standards and guidelines Ensuring successful transitions for solutions into production ensuring production support have the necessary knowledge and documentation to support the service. Skills, knowledge and experience Robust Data and Technical/Solutions Architecture skills - sets direction for and possesses a deep understanding of architecture and strategies which integrate with industry trends e.g. TOGAF Hands-on experience with analytical tools and languages such as: Python, R, SQL, Azure Data Factory (and SSIS), Databricks, Power BI, Git etc. Experience of infrastructure: cloud-based technologies used for storage, data lakes, data warehouses, data streaming, Databases, ETL / ELT / Transformation. Experience of DevOps/ DataOps methods in development of data solutions to ensure pipelines and processes can be automated and controlled. Experience with Cloud based data and analytical initiatives utilising Azure, AWS, Google Cloud or similar cloud services Experience of working closely with Data Professionals (E.g. Data Scientists and Data Analysts) to understanding their needs. Experience of implementating of statistical, Artificial Intelligence, Machine Learning and Deep learning applications. Experience with integrations (e.g. via APIs) with external vendors to share data between organizations Experience of working with external technology supplier and service providers to deliver business solutions SFIA Skill Enterprise and business architecture STPL - Level 5 Solution architecture ARCH - Level 5 Requirements definition and management REQM- Level 5 Database design DBDS- Level 5 Analytics INAN- Level 4 Emerging Technology Monitoring (EMRG)- Level 4 Relationship Management RLMT- Level 5
Ampersand Consulting
Ampersand Consulting
Senior Data Engineer (Cloud, Azure, SQL, Python, Java) The UK's leading provider of IT solutions and services is looking to hire an experienced Senior Data Engineer (Cloud, Azure, SQL, Python, Java) to join their rapidly expanding Data Analytics team working closely with the Chief Data Scientist based in Central London. The Senior Data Engineer (Cloud, Azure, SQL, Python, Java) will implement complex multi/hybrid cloud based big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple technology platforms. The Senior Data Engineer (Cloud, Azure, SQL, Python, Java) will be a technical contributor with hands-on knowledge of all phases in building large-scale cloud based distributed data processing systems and applications. Responsibilities for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) Lead the design, implementation, and continuous delivery of pipelines using distributed Azure based big data technologies supporting data processing initiatives across batch and streaming datasets Experience in hybrid cloud solutions and experience in GCP resources Responsible for development using Scala, Python languages and Big Data Frameworks such as Spark, EMR, Kafka, Storm, Jenkins, Jfrog Artifactory and DataBricks Provide administrative support on deployed Azure platform components Identify, evaluate and implement cutting edge big data pipelines and frameworks required to provide requested capabilities to integrate external data sources and APIs Review, analyse and evaluate market requirements, business requirements and project briefs in order to design the most appropriate end-to-end technology solutions Process and manage high volume real time customer interaction streams Provide architectural support by building Proof of Concepts & Prototypes Self-Starter to deliver data engineering solutions to optimize both the cost and existing solution Requirements for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) Extensive Software Industry experience Development experience with Azure services (such as Data Factories, Data Lake Storage, SQL Elastic Pools, Data Pipeline, DataBricks, Kubernetes Services, Apache Nifi, jFrog etc) Development experience with GCP services (Cloud Storage, Cloud Spanner, BigQuery) Experience with Apache Spark and NoSQL Implementation Extensive working knowledge in different programming or scripting languages like Scala, Java, Linux, Shell, SQL, Python Proficiency working with structured, semi-structured and unstructured data sets including social, web logs and real time streaming data feeds Able to tune Big Data solutions to improve performance and end-user experience Knowledge on Visualization and Data Science Tools Expert level usage with Jenkins, GitHub is preferred If you would like to apply for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) role then please click the apply button
Feb 15, 2019
Ampersand Consulting
Ampersand Consulting
Senior Data Engineer (Cloud, Azure, SQL, Python, Java) The UK's leading provider of IT solutions and services is looking to hire an experienced Senior Data Engineer (Cloud, Azure, SQL, Python, Java) to join their rapidly expanding Data Analytics team working closely with the Chief Data Scientist based in Central London. The Senior Data Engineer (Cloud, Azure, SQL, Python, Java) will implement complex multi/hybrid cloud based big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple technology platforms. The Senior Data Engineer (Cloud, Azure, SQL, Python, Java) will be a technical contributor with hands-on knowledge of all phases in building large-scale cloud based distributed data processing systems and applications. Responsibilities for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) Lead the design, implementation, and continuous delivery of pipelines using distributed Azure based big data technologies supporting data processing initiatives across batch and streaming datasets Experience in hybrid cloud solutions and experience in GCP resources Responsible for development using Scala, Python languages and Big Data Frameworks such as Spark, EMR, Kafka, Storm, Jenkins, Jfrog Artifactory and DataBricks Provide administrative support on deployed Azure platform components Identify, evaluate and implement cutting edge big data pipelines and frameworks required to provide requested capabilities to integrate external data sources and APIs Review, analyse and evaluate market requirements, business requirements and project briefs in order to design the most appropriate end-to-end technology solutions Process and manage high volume real time customer interaction streams Provide architectural support by building Proof of Concepts & Prototypes Self-Starter to deliver data engineering solutions to optimize both the cost and existing solution Requirements for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) Extensive Software Industry experience Development experience with Azure services (such as Data Factories, Data Lake Storage, SQL Elastic Pools, Data Pipeline, DataBricks, Kubernetes Services, Apache Nifi, jFrog etc) Development experience with GCP services (Cloud Storage, Cloud Spanner, BigQuery) Experience with Apache Spark and NoSQL Implementation Extensive working knowledge in different programming or scripting languages like Scala, Java, Linux, Shell, SQL, Python Proficiency working with structured, semi-structured and unstructured data sets including social, web logs and real time streaming data feeds Able to tune Big Data solutions to improve performance and end-user experience Knowledge on Visualization and Data Science Tools Expert level usage with Jenkins, GitHub is preferred If you would like to apply for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) role then please click the apply button
Ampersand Consulting
Senior Data Engineer (Cloud, Azure, SQL, Python, Java) The UK's leading provider of IT solutions and services is looking to hire an experienced Senior Data Engineer (Cloud, Azure, SQL, Python, Java) to join their rapidly expanding Data Analytics team working closely with the Chief Data Scientist based in Central London. The Senior Data Engineer (Cloud, Azure, SQL, Python, Java) will implement complex multi/hybrid cloud based big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple technology platforms. The Senior Data Engineer (Cloud, Azure, SQL, Python, Java) will be a technical contributor with hands-on knowledge of all phases in building large-scale cloud based distributed data processing systems and applications. Responsibilities for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) Lead the design, implementation, and continuous delivery of pipelines using distributed Azure based big data technologies supporting data processing initiatives across batch and streaming datasets Experience in hybrid cloud solutions and experience in GCP resources Responsible for development using Scala, Python languages and Big Data Frameworks such as Spark, EMR, Kafka, Storm, Jenkins, Jfrog Artifactory and DataBricks Provide administrative support on deployed Azure platform components Identify, evaluate and implement cutting edge big data pipelines and frameworks required to provide requested capabilities to integrate external data sources and APIs Review, analyse and evaluate market requirements, business requirements and project briefs in order to design the most appropriate end-to-end technology solutions Process and manage high volume real time customer interaction streams Provide architectural support by building Proof of Concepts & Prototypes Self-Starter to deliver data engineering solutions to optimize both the cost and existing solution Requirements for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) Extensive Software Industry experience Development experience with Azure services (such as Data Factories, Data Lake Storage, SQL Elastic Pools, Data Pipeline, DataBricks, Kubernetes Services, Apache Nifi, jFrog etc) Development experience with GCP services (Cloud Storage, Cloud Spanner, BigQuery) Experience with Apache Spark and NoSQL Implementation Extensive working knowledge in different programming or scripting languages like Scala, Java, Linux, Shell, SQL, Python Proficiency working with structured, semi-structured and unstructured data sets including social, web logs and real time streaming data feeds Able to tune Big Data solutions to improve performance and end-user experience Knowledge on Visualization and Data Science Tools Expert level usage with Jenkins, GitHub is preferred If you would like to apply for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) role then please click the apply button
Feb 15, 2019
Ampersand Consulting
Senior Data Engineer (Cloud, Azure, SQL, Python, Java) The UK's leading provider of IT solutions and services is looking to hire an experienced Senior Data Engineer (Cloud, Azure, SQL, Python, Java) to join their rapidly expanding Data Analytics team working closely with the Chief Data Scientist based in Central London. The Senior Data Engineer (Cloud, Azure, SQL, Python, Java) will implement complex multi/hybrid cloud based big data projects with a focus on collecting, parsing, managing, analyzing and visualizing large sets of data to turn information into insights using multiple technology platforms. The Senior Data Engineer (Cloud, Azure, SQL, Python, Java) will be a technical contributor with hands-on knowledge of all phases in building large-scale cloud based distributed data processing systems and applications. Responsibilities for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) Lead the design, implementation, and continuous delivery of pipelines using distributed Azure based big data technologies supporting data processing initiatives across batch and streaming datasets Experience in hybrid cloud solutions and experience in GCP resources Responsible for development using Scala, Python languages and Big Data Frameworks such as Spark, EMR, Kafka, Storm, Jenkins, Jfrog Artifactory and DataBricks Provide administrative support on deployed Azure platform components Identify, evaluate and implement cutting edge big data pipelines and frameworks required to provide requested capabilities to integrate external data sources and APIs Review, analyse and evaluate market requirements, business requirements and project briefs in order to design the most appropriate end-to-end technology solutions Process and manage high volume real time customer interaction streams Provide architectural support by building Proof of Concepts & Prototypes Self-Starter to deliver data engineering solutions to optimize both the cost and existing solution Requirements for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) Extensive Software Industry experience Development experience with Azure services (such as Data Factories, Data Lake Storage, SQL Elastic Pools, Data Pipeline, DataBricks, Kubernetes Services, Apache Nifi, jFrog etc) Development experience with GCP services (Cloud Storage, Cloud Spanner, BigQuery) Experience with Apache Spark and NoSQL Implementation Extensive working knowledge in different programming or scripting languages like Scala, Java, Linux, Shell, SQL, Python Proficiency working with structured, semi-structured and unstructured data sets including social, web logs and real time streaming data feeds Able to tune Big Data solutions to improve performance and end-user experience Knowledge on Visualization and Data Science Tools Expert level usage with Jenkins, GitHub is preferred If you would like to apply for the Senior Data Engineer (Cloud, Azure, SQL, Python, Java) role then please click the apply button