Removing docker dependency possible?

[AnneBeyer]

We have some problems with the docker dependency, and I was wondering whether it would make sense to replace the lm-zoo dependency with plain python modules and access the models directly. I'm mostly working with models that are part of huggingface's transformers, so I'm not sure about the implications this might have on other models, but I'd be happy to hear your thoughts.

[hans]

Hi Anne, thanks for using SyntaxGym! We set up the Docker pipeline in order to uniformly support a few different models that were implemented using different software stacks. But I understand your use case may be different if you're just using huggingface models.

While we don't have the development cycles in our lab to support this level of structural change, if you're interested, I don't think it should be too extreme of a lift. The easiest way would be to implement a new "backend" within lm-zoo -- i.e., an abstract interface that supports running the typical model commands like tokenize and get_surprisals. The backend abstraction is already in place to support running models either via Docker or via Singularity, and you could add a Huggingface backend potentially!

Some relevant code pointers if you're interested within lm-zoo:

• abstract backend definition https://github.com/cpllab/lm-zoo/blob/master/lm_zoo/backends/__init__.py#L7
• an example model invocation for get_surprisals, which gets routed through a relevant backend. Notice the backends assume some shell interface -- the backend receives a command like get_surprisals /dev/stdin -- but this could be generalized to support Python APIs, too. https://github.com/cpllab/lm-zoo/blob/master/lm_zoo/__init__.py#L90

---

Alternatively -- if you tell me more about the Docker issues you're experiencing I can try to be helpful there. (Or is this the GPU issue you already posted about?)

[AnneBeyer]

Thank you for your response! The main problem is that our GPU infrastructure itself is already based on docker, which makes it a challenge to get the lm-zoo to run inside a container in the first place. The backend option does sound interesting, I'll look into that and get back to you if I have any questions. Thank you!

[hans]

I see. For what it's worth, Docker-in-Docker should definitely work for lm-zoo / syntaxgym -- I've done this before in our own applications.

[hans]

Hi Anne, I don't know if you've made progress on this? Just FYI, I'm beginning a refactor now within lm-zoo (and later SyntaxGym) to support Model definitions that are defined Pythonically, without reference to Docker.

[AnneBeyer]

Hi Jon, great, thank you for letting me know! I haven't had the time to look into this yet, sorry. Let me know if I can help somehow.