About Scrumconnect
Scrumconnect Consulting is a UK-based digital transformation consultancy delivering agile, secure technology solutions for public and private sector clients. This is a fully remote role based in India, supporting our UK operations and client base. You will work closely with our UK leadership team and must be comfortable operating within UK regulatory and legal frameworks.
About the Role
We are looking for a Test Engineer to join a large-scale cloud data engineering programme, operating across a modern AWS-native technology stack including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, and DynamoDB.
You will own testing across automated pipelines, data workflows, and cloud infrastructure - identifying risks, championing test frameworks, and coaching colleagues in quality engineering practices.
This is a hands-on, technically deep role. You will write code, build and extend test frameworks, and take accountability for test quality across releases, while actively mentoring junior team members.
Key Responsibilities
Automated Testing
Design, implement, and maintain automated test suites across data pipelines, cloud services, and applications
Automate data and application testing tasks and build test coverage using existing or new infrastructure
Risk Identification & Reporting
Identify and raise awareness of risks arising from automated test results
Analyse and report on test activities, results, issues, and risks
Translate technical findings into clear, prioritised communication
Framework Development
Identify, evaluate, and implement new test frameworks
Enhance existing frameworks to improve testing confidence and coverage
Pipeline & Data Testing
Validate data pipelines using Apache Airflow, AWS Glue, Athena, and EMR
Ensure data integrity, transformation accuracy, and performance under load
Analyse data in multiple formats to validate new functionality
Perform production data analysis to identify root causes of issues
Mentoring & Coaching
Mentor and advise team members on testing best practices
Contribute to a quality-first engineering culture
CI/CD & DevOps
Integrate and manage automated tests within GitLab CI/CD pipelines
Ensure automated execution and fast feedback loops
Technology Stack
Apache Airflow, Amazon Athena, AWS S3, AWS Glue, AWS EMR, AWS EC2, AWS ECR, AWS DynamoDB, AWS CloudWatch, AWS IAM, Python, Java, SQL, Bash, GitLab CI/CD, Jupyter Notebooks, Apache Spark, Terraform, Docker
Key Skills
Strong proficiency in Python, Java, SQL, and Scripting (eg, Bash)
Experience with Apache Airflow for orchestration and log analysis
Hands-on experience with AWS services (S3, Glue, Athena, EMR, CloudWatch, IAM, DynamoDB, EC2, ECR)
Experience using GitLab and GitLab CI/CD pipelines
Ability to analyse data using Jupyter Notebooks and Amazon Athena
Understanding of Apache Spark and EMR (basic to intermediate)
Experience with infrastructure as code (Terraform)
Familiarity with Docker and containerization
Knowledge of Server Side and client-side encryption
Understanding of dimensional data models and slowly changing dimensions
Experience in data creation/generation for testing
Strong analytical skills for validating data and identifying issues
Ability to understand how customer requirements translate into functional solutions
Skills & Experience Required
Proven experience as a Test Engineer or QA Engineer in cloud/data environments
Strong Python skills for automation, data validation, and test development
Experience testing data pipelines (data quality, transformation, performance)
Experience designing or extending automated test frameworks
Strong problem-solving and root cause analysis skills
Ability to clearly communicate risks to technical and non-technical stakeholders
Experience mentoring or coaching junior engineers
SFIA Level 4 capability - works autonomously and influences others
Active SC clearance (mandatory)
Diversity and Inclusion
At Scrumconnect Consulting, we believe that diversity drives innovation. We are committed to creating an inclusive environment where every individual is respected, valued, and supported. We welcome applications from candidates of all backgrounds and experiences.