Should ModelPack standardize a Modelfile format?

[ericcurtin]

Background

A contributor recently opened docker/model-runner#903 to add Modelfile support to Docker Model Runner. The implementation follows Ollama's Modelfile format, which is oriented around runtime configuration (inference parameters, prompt templates, system prompts, etc.).

At the same time, modctl ships its own Modelfile format (via modctl modelfile generate), which serves a different purpose: describing packaging metadata for OCI artifacts (architecture, format, precision, weights, configs, etc.).

This gives us two things both called "Modelfile":

	modctl Modelfile	Ollama Modelfile
Purpose	OCI packaging metadata	Runtime/inference configuration
Key fields	NAME, ARCH, FORMAT, MODEL, CONFIG	FROM, PARAMETER, TEMPLATE, SYSTEM

Questions for the community

Does ModelPack plan to formally specify a Modelfile format as part of this spec?

[bergwolf]

The modelpack spec is to describe a model packaged in OCI format. Modelfile in modctl's case is to describe how to build the artifacts, while Modelfile in Ollama's case is for runtime to consume it. Right now they are actually two different files serving different purpose. Are you concerning the filename conflict or do you see some issue with the two Modelfiles?

On specifying a Modelfile format as part of this spec, IMO we should be open to it. Right now, there are multiple packaging tools (modctl/AIKit/KitOps). If there are interests in defining a common Modelfile format, we should go for it.

[ericcurtin]

The modelpack spec is to describe a model packaged in OCI format. Modelfile in modctl's case is to describe how to build the artifacts, while Modelfile in Ollama's case is for runtime to consume it. Right now they are actually two different files serving different purpose. Are you concerning the filename conflict or do you see some issue with the two Modelfiles?

I'm not concerned about the conflict.

On specifying a Modelfile format as part of this spec, IMO we should be open to it. Right now, there are multiple packaging tools (modctl/AIKit/KitOps). If there are interests in defining a common Modelfile format, we should go for it.

This is what I'm interested in, a CNCF/ModelPack defined standard. Ollama do lots of non-standard things, non-standard OCI artifacts, non-standard OCI transport, non-standard gguf files. So I'm interested in a CNCF/ModelPack defined standard of Modelfile.

[gorkem]

@ericcurtin The naming collision is unfortunate but worth separating from the question of scope.
Kitfiles in KitOps are packaging manifests, not runtime configs. The same is true for modctl's Modelfile. Ollama's Modelfile is a different artifact entirely: it describes how to run a model, not how to package one.

ModelPack is the packaging spec. It is not going to define a runtime config format and it should not be called Modelfile if it ever did.

What ModelPack can do, and arguably should, is extend the artifact's own metadata to carry runtime hints (chat template, recommended context length, stop tokens, etc.) as part of the config descriptor or annotations. Runtimes that consume ModelPack artifacts can then configure themselves directly from the artifact, without a separate Modelfile of either flavor. That is a different proposition from standardizing a Modelfile syntax.

Is the runtime-properties direction what you are looking for, or were you asking about a packaging Modelfile?

[ericcurtin]

@ericcurtin The naming collision is unfortunate but worth separating from the question of scope. Kitfiles in KitOps are packaging manifests, not runtime configs. The same is true for modctl's Modelfile. Ollama's Modelfile is a different artifact entirely: it describes how to run a model, not how to package one.

ModelPack is the packaging spec. It is not going to define a runtime config format and it should not be called Modelfile if it ever did.

Ok lets say this variant is called Llmfile for the sake of it (I'm just trying to avoid the naming discussion, but I'm genuinely happy with any old name).

What ModelPack can do, and arguably should, is extend the artifact's own metadata to carry runtime hints (chat template, recommended context length, stop tokens, etc.) as part of the config descriptor or annotations. Runtimes that consume ModelPack artifacts can then configure themselves directly from the artifact, without a separate Modelfile of either flavor. That is a different proposition from standardizing a Modelfile syntax.

Is the runtime-properties direction what you are looking for, or were you asking about a packaging Modelfile?

I'm open minded, in DMR we have "docker model package various args". But I think a standardized "Llmfile" would be worth it. And if anything if gives some level familiarity with ollama, which users crave. Now ours can be completely different to Ollama's Modelfile in the content required, but I think a standard is worth it

[ericcurtin]

# Llmfile — runtime inference configuration

# Required: OCI model reference (ModelPack artifact or registry tag)
FROM smollm2:latest

# Optional: operational mode (default: completion)
MODE completion  # completion | embedding | reranking | image-generation

# Optional: system prompt
SYSTEM You are a helpful, concise assistant.

# Optional: Jinja2 chat template (overrides model default)
TEMPLATE """
{{- if .System }}<|system|>
{{ .System }}<|end|>
{{- end }}
<|user|>
{{ .Prompt }}<|end|>
<|assistant|>
"""

# Optional: runtime parameters
PARAMETER context_size 8192
PARAMETER keep_alive 5m
PARAMETER think false

# Speculative decoding
PARAMETER speculative_draft_model ai/smollm2-draft:latest
PARAMETER speculative_num_tokens 5
PARAMETER speculative_min_acceptance_rate 0.9

[ericcurtin]

@areebahmeddd @ilopezluna @doringeman thoughts?

[ilopezluna]

I like the idea of standardizing a ModelPack "Llmfile".

My main question is whether this file is intended to be a one-to-one representation of a ModelPack OCI artifact:

• From an Llmfile, can I deterministically create one OCI artifact?
• From an OCI artifact, can I recover the equivalent Llmfile?
• If yes, does the Llmfile become another representation of the model artifact itself?

I think this could make adoption of ModelPack easier. Consumers would not need to construct OCI artifacts using random parameters, ad hoc conventions, or tool-specific mechanisms. Instead, they could consume a well-defined file format and map it to an OCI artifact. That would also make it easier to build SDKs around the standard.

Another benefit is that files are easy to version, review, diff, and store in source control, which could help make model packaging workflows more reproducible.

[doringeman]

Hello everyone,

+1 on a standard runtime config file. I like the idea of having some parameters, such as chat template, stop tokens, context part of the artifact, and the file layers runtime overrides on top (system prompt, speculative decoding, etc.).
I see the file (Llmfile or whatever) as what you'd use in Compose, as how you run the model; the artifact is the model.