Feature : pull only parts of a repository

[apraga]

Hi,

In my experiments, data can be quite large. It would be awesome to have a way to pull only some files or subfolders to avoid taking too much space. For example, working on the output files of an experiments would be improved if one did not have to download the raw data to be processed.

Thanks,

[vsbuffalo]

Sure, I think this a great idea and something other users have requested. I think the main thing is to figure out how best particular subsets of the data should be downloaded. So I can best design this upcoming feature, what did you envision? E.g. would you use sdf get file_1 file_2 etc? or have some sort of "tag" that you could use sdf <tag>?

[apraga]

Thanks for the quick feedback ! My first idea was to follow the model of git add where a list of files or folders is supplied. If a folder is given, all files and subdirectories should be included.

This could be passed as optional arguments to sdf pull (that feels the most natural at the moment). What do you think ?

[apraga]

@vsbuffalo If that's okay with you, I would like to take a stab at it.