Data Engineer (Lumkani)

About

Description

We are developing and refining our existing IoT platform, policy management system, and agent app. To support these efforts, we need to migrate and improve our business intelligence (BI) infrastructure. As a Data Engineer, you will take ownership of this migration, focusing on designing and building scalable data pipelines that will support our BI and analytics needs. In this role, you will be responsible for evaluating architectures and technologies, building proofs of concept (PoCs), and driving the development of ETL processes that will enhance our data infrastructure. You’ll work closely with other team members to ensure the systems you develop are cohesive, maintainable, and scalable, aligning with the broader goals of the organisation. Our current stack includes AWS, Django, and Vue.js. We leverage the power of Python for many of our backend services, and rely on the scalability and robustness of cloud technologies to handle our growing data needs. Your work will focus on developing solutions that can efficiently handle the data generated by our platform, which manages an IoT system of fire detectors deployed in informal settlements across South Africa, as well as managing insurance agents who offer innovative household insurance products in these communities. This position will require you to design, deploy, and maintain features within the data infrastructure that support critical functions like payment processing, BI reporting, and agent management. You’ll work on ETL pipelines, data modelling, and storage solutions that will ensure we can make sense of the data generated by our platform. This is an exciting role that offers the opportunity to work with a diverse range of technologies, take ownership of key data projects, and make a real impact within a passionate and inclusive team. Your work will contribute significantly to life-saving technology and help drive financial inclusion for underserved communities in South Africa and beyond.

Responsibilities

• Design and Develop Data Pipelines – Build reliable, scalable pipelines for ingesting, transforming, and storing data
• Optimize Data Models – Design well-structured and efficient schemas for analytics and reporting
• Ensure Data Quality – Monitor, validate, and enforce data integrity and consistency throughout the system
• Write Clean and Maintainable Code – Emphasize readability, performance, and scalability in development practices
• Implement Testing Frameworks – Create and maintain unit and integration tests to validate data workflows
• Leverage AWS Services – Use AWS tools effectively for storage, computation, and transformation tasks
• Collaborate with the Team – Work closely with other engineers, share knowledge, and follow team development practices
• Research and Innovate – Stay current with data engineering trends and continuously seek improvements in tooling and architecture

Qualifications

• Experience in Data Engineering – 6+ years of professional experience in designing and implementing data solutions
• Expertise in Data Warehousing – Hands-on experience with modern data warehousing solutions (e
• g
• , Redshift, Snowflake)
• Strong Data Modelling Skills – Proven ability to create efficient, scalable, and optimized data structures
• Proficiency in ETL and Data Pipelines – Experience designing and maintaining ETL processes using modern tools and practices
• Programming Skills in Python – Strong command of Python and its data-focused libraries (e
• g
• , Pandas, PySpark)
• Advanced SQL and Relational Database Knowledge – Deep understanding of relational databases such as MySQL and PostgreSQL
• Experience with AWS Ecosystem – Comfortable using AWS services such as S3, EC2, Glue, and Redshift in data workflows
• Strong Soft Skills – Detail-oriented, collaborative, self-motivated, and committed to code quality and continuous learning

Application instructions:

• click the apply button to apply