Chain of Demand
The Data Engineer will be responsible for the delivery of data solutions in a timely manner. They will report directly to Head of Data Science, and will work closely with other areas of the business such as Technology and Management.
The role will suit individuals that are flexible and creative, enjoy solving complex data problems. The right candidate will be looking to take ownership of the data engineering technology stack and all related processes and procedures.
• Design, build, and maintain data structures, databases, and data processing pipelines to support data science projects
• Build and maintain web scrapers, APIs, and dashboards
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources:
o Handle different types of data
o Ensure data quality by placing strong controls in place
o Automate management data loads, scheduling of batch jobs including logging/alerts and actions
• Pre-process data ready for analysis
• Support and practice CI / CD principles
• Maintenance of development standards and procedures
• The candidate must have proven relevant data engineering experience
• Must have MongoDB experience
• Strong experience in scraping websites and other platforms• Ability to code in Python to create web scrapers, ETL processes and data analysis pre-processes
• Previous experience or knowledge of handling large datasets
• Knowledge of CI/CD and GIT principles
• Strong team player, whilst also able to work independently as required and take ownership of projects• Flexible and adaptable
• Technology enthusiast, intellectually curious and problem solver
• Ability to work under pressure, give realistic estimates, and work to deadlines
• Strong written and verbal communication skills
Chain of Demand uses AI and predictive analytics to help companies and investors make sense of data, so they can take business decisions to the next level.
Official Website: https://www.chainofdemand.co/