Extract, Transfer, Load (ETL) projects
You might think that moving some data around from A to B should be simple. Unfortunately it is not that straight forward. Having done a number of ETL projects, I’ve had to deal with:
- Learning new APIs, including Amazon MWS, AppFigures, Square, TicketSocket Enterprise and Premiere, HubSpot and Google BigQuery
- For each API, handling authentication and authorisation, and testing using a sandbox environment (if possible) or on live data (if necessary)
- Tracking what is already transferred, in case of failure or to facilitate a regular process
- Extracting what still needs transferring. Somewhat surprisingly, this is not always well catered for
- Running and controlling the ETL process, for instance as a Flask application or a Singer Tap
- Providing an efficient initial data load and a robust ongoing transfer
Some of my ETL projects have come from the UpWork contract market place and can be seen on my UpWork profile. These all gained a 5 star review and some very positive feedback. For instance, one client said “Coen is a gem! He is proactive, reads requirements thoroughly, investigates challenges, and has excellent communication habits and skills. He was able to build out a tool that we needed for months in about a week”
Contact me if you’ve got some data that needs moving
