Everyone doing professional ML needs to deal with setting up ETL"work-flows" or processing "pipelines" where at some time.
Example of workflow tool features:
- Schedule processing jobs to run at a specific LOCATION and TIME
- Take further action (eg. trigger next job) depending on the status of the
- Handle failure of job (eg. time-out, invalid data) or machine (eg. not accessible)
- Have a web-interface for easy setup and management of jobs
- Manage user-accounts and enable resource sharing
What are the r/ML community's favorite workflow tools (especially if open-source)?
[link][4 comments]