Job Title:  Data Engineer

Job Summary

As a Data Engineer, you will be part of a team responsible for designing, building, and maintaining data pipelines, databases, and data infrastructure. You will work with a cross-functional team, collaborating closely with data scientists, analysts, and DevOps teams to ensure that data is available, accurate, and optimized for performance.  

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for ingesting, processing, and storing large datasets from various sources.
• Implement ETL (Extract, Transform, Load) processes to ensure data is properly cleaned, transformed, and stored in data warehouses or data lakes.
• Manage and optimize databases, data warehouses, and data lakes to ensure data is stored efficiently and securely.
• Utilize cloud-based data platforms (e.g., AWS, Azure, Google Cloud) to build and maintain scalable and secure data infrastructure.
• Implement data quality checks and validation processes to ensure the accuracy and consistency of data.
• Ensure compliance with data governance standards, including data privacy, security, and regulatory requirements.
• Manage metadata and ensure proper data lineage, enabling accurate data tracking and auditing.
• Assist in data modeling and schema design to ensure data is structured effectively for analysis and reporting.
• Continuously review and refine data engineering practices, tools, and processes to enhance efficiency, quality, and scalability.
• Contribute to the automation of repetitive tasks, improving the overall efficiency of the data engineering function.
   

Skills & Competencies

• Work experience programming in Python
• Strong SQL skills and experience with database management
• Familiarity with data processing frameworks (e.g., Apache Spark, Kafka) and ETL tools
• Workloads Understanding of cloud services such as EC2, S3, Lamba (AWS), or their equivalents in other cloud providers
• Knowledge of data warehousing solutions (e.g., Snowflake, Redshift, BigQuery).
• Strong scripting skills (Bash, Python) for automating tasks and process
• Familiarity with CI/CD pipelines and version control systems (e.g., Git).

Relevant Experiences & Qualifications

• A minimum of 3 years of experience in a related field
• Bachelor’s degree in computer science, computer engineering, or a related field
• Certifications in relevant technologies (e.g., AWS Certified  Data Analytics, Google Professional Data Engineer) are an advantage
   

Closing Date

Monday 28th October 2024, 5pm