• Develop and maintain data pipelines using Python, SQL, and Apache Airflow.• Containerise data applications and workflows using Docker.• Build and optimise data solutions using AWS services including:• Redshift, OpenSearch, Lambda, Glue, Step Functions, and Batch.• Collaborate with cross-functional teams to deliver robust, secure, and scalable data infrastructure.• Manage version control using GitLab (or similar).• Monitor and troubleshoot data pipeline performance and reliability.• Contribute to documentation and process improvement initiatives.• Ensure data quality, governance, and security best practices are followed.
Requirements
- Strong programming skills in Python and proficiency in SQL.
- Experience with Apache Airflow for DAG orchestration and monitoring.
- Hands-on experience with Docker for containerisation.
- Proficient in AWS data services:
- Redshift, OpenSearch, Lambda, Glue, Step Functions, Batch.
- Familiarity with CI/CD pipelines and YAML-based configuration (e.g., GitLab CI/CD).
- Proficient in Git and collaborative development using GitLab (or similar).
- Understanding of AWS security best practices, IAM policies, and RBAC.
- Desirable Skills:
- Experience with AWS services such as Athena, SQS, CloudWatch, CloudTrail, EMR.
- Exposure to infrastructure-as-code tools (e.g., Terraform, CloudFormation).
- Familiarity with documentation tools like Confluence and README standards.
- Experience working in consulting or client-facing environments.
- Strong problem-solving and troubleshooting abilities.
- Comfortable working in Agile environments.
- Effective stakeholder management and communication skills.
- Collaborative mindset and willingness to share knowledge.
ATS Optimization KeywordsBelow are skills and terms extracted directly from this job posting to improve Applicant Tracking System (ATS) visibility. This unique feature helps candidates tailor their applications more effectively — a feature exclusive to JobTailor job listings.
Hard Skills
- Python
- SQL
- Apache Airflow
- Docker
- AWS Redshift
- AWS OpenSearch
- AWS Lambda
- AWS Glue
- AWS Step Functions
- AWS Batch
Soft Skills
- problem-solving
- troubleshooting
- stakeholder management
- communication
- collaboration
- process improvement
- data governance
- data quality
- Agile mindset
- cross-functional teamwork
#J-18808-Ljbffr…