Basically I have a problem which is more related to the internal structure of different ML/stats projects rather than ML per se.
I have quite a few different projects. which are always under ~/Projects/project1/ ~/Projects/project2/ etc.
For consistency I like all of the project dirs having the same structure, like "inputs", "outputs", "meta", "src". Or something very similar.
But here is the problem - ideally I would like outputs being produced by doing something to a subset of inputs. However many times I need to save intermediate results to outputs (since they take a long time to compute) and then use those. So the flow starts to go from outputs to outputs. (which irks me a little, because every time I change something it becomes harder to track which parts need to be redone.
In addition I like doing some kind of presentation/graphics/description of the results each time I finish some part. And I am having trouble deciding where to put those. One idea is associate every output folder with some kind of result - which makes the individual folders of outputs more complicated and unique. Another possibility is having a separate folder like "results". But computing something in output and then having to do something in a different folder for "presentation" also bothers me a little. And what's worse is that "results" folder get's cluttered really fast in practice, if the project is running for a longer period of time.
So this is my rant and some minor issues which for some reason effect the productivity of what I do. It seems important yet I can find no literature or any suggestions to formally address these things.
Can someone with a similar situation share how they are organizing their work and what works/does not work in practice?
Or better yet - suggest some literature (if there is any) addressing this?
EDIT: Seems like this thread quickly got nowhere, I guess people are not into these kinds of things. I will just quickly share some of the resources that I found useful for anyone who might stumble over this post in the future:
http://arkitus.com/patterns-for-research-in-machine-learning/
https://news.ycombinator.com/item?id=4384317
http://www.ploscompbiol.org/article/info%3Adoi%2F10.1371%2Fjournal.pcbi.1000424
http://www.theexclusive.org/2012/08/principles-of-research-code.html
[link][14 comments]