Senior Data Engineer - Data Products & AI
Beacon Intelligence London (Hybrid) Salary: £90,000 + Bonus
At Beacon Intelligence, we are a SaaS data and analytics business focused on delivering high-value insights through our proprietary platform. Data sits at the core of everything we do - powering our products, driving customer value, and enabling scalable growth.
We have built a modern data platform over the last 24 months, consisting of Airbyte as our ETL tool and Databricks as our scalable data processing layer, all hosted on Azure Cloud Services. This platform underpins our product offering and is central to our continued evolution as a data-driven SaaS business.
The Role
As a Senior Data Engineer (Data Products & AI), you will play a key role in building and scaling Beacon's data products.
This is not a traditional data engineering role focused purely on pipelines. Instead, you'll work at the intersection of data, product, and commercial teams, turning datasets into customer-facing, revenue-generating solutions.
You'll combine strong engineering expertise with a data product mindset, structuring, enriching, and operationalising data through modern platforms and AI systems.
Key Responsibilities Data Product Development
- Design and build scalable data products that underpin Beacon's commercial offerings
- Transform raw and third-party data into structured, enriched, product-ready datasets
- Partner with product and commercial teams to define how data is packaged, accessed, and monetised
- Deliver data via APIs, internal tools, and customer-facing platforms
AI & Data Enrichment
- Apply AI/LLM capabilities to enhance datasets (e.g. classification, tagging, summarisation, insight generation)
- Build and integrate LLM-powered product features
- Implement RAG frameworks to improve data quality and unlock new functionality
- Support development of AI-driven products (search, recommendations, insight tools)
- Ensure AI systems are scalable, governed, and production-ready
Platform & Engineering
- Build and maintain robust data pipelines (batch and streaming)
- Own and evolve the data platform (Databricks, Azure, Airbyte)
- Ensure high standards of data quality, testing, and observability
- Improve efficiency through automation, CI/CD, and engineering best practices
- Support deployment and lifecycle management of data and AI systems
Commercial & Stakeholder Impact
- Collaborate with senior stakeholders to identify high-value data opportunities
- Translate business requirements into scalable data and AI solutions
- Act as a bridge between technical and non-technical teams
- Contribute to prioritisation based on commercial impact
Leadership & Growth
- Influence the evolution of Beacon's data product and AI strategy
- Mentor junior engineers and promote strong engineering standards
- Support the development of a high-performing, commercially focused data team
What We're Looking For Core Engineering Experience
- Strong experience with modern data platforms (e.g. Databricks) and cloud (Azure preferred)
- Advanced Python and SQL, with a focus on scalable, production-quality solutions
- Experience with Apache Spark (PySpark) for large-scale data processing
- Proven experience building data pipelines and data models
- Experience working with APIs and data delivery mechanisms
- Familiarity with CI/CD, Git, and modern development practices
Data Product Mindset
- Experience building or supporting data products or customer-facing data solutions
- Strong understanding of how to structure and expose data to end users
- Ability to think beyond pipelines and focus on commercial value
- Experience working closely with product or commercial teams
AI / LLM Experience
- Hands on experience or strong familiarity with LLMs (e.g. Azure OpenAI, LangChain)
- Experience applying AI to real-world use cases
- Experience working with unstructured data (text, documents, web data)
- Understanding of embeddings, vector databases, or semantic search
- Awareness of AI governance and data ethics
Desirable Experience
- Experience with Airbyte or modern data ingestion tools
- Infrastructure-as-code (e.g. Terraform)
- Exposure to LLM frameworks (LangChain, LlamaIndex)
- Experience with experimentation or A/B testing
- Background building AI-enabled or data-driven products
- Life sciences or healthcare domain experience
How You Work
- Product-focused - you think in terms of outcomes, not just pipelines
- Commercially aware and impact-driven
- Pragmatic and delivery-oriented
- Collaborative, with strong stakeholder engagement skills
- Curious about AI and emerging technologies, with a focus on scalable implementation
Why Join Beacon Intelligence
- Build products, not just infrastructure, with real-world impact on patients
- Help shape how AI is embedded into a growing SaaS/data platform
- High-impact role with influence across product and commercial strategy
- Work with a modern, evolving tech stack
- Strong ownership and growth opportunities
- Collaborative environment with teams that value data and insight