Q: Can GPU Operator + MPS split an ADA6000 into two memory partitions (30 GB + 18 GB) for vLLM?

[flobrunner]

Hi,

I have a requirement to run two models on a single NVIDIA ADA6000 GPU using the GPU Operator and MPS (Multi-Process Service):

• One model requires ~30 GB of GPU memory.
• The other model requires ~18 GB of GPU memory.
• Both models will be started with [vLLM](https://github.com/vllm-project/vllm).

I’d like to know if it’s possible to configure MPS via the GPU Operator so that the GPU can be split into these two “memory slices” (30 GB + 18 GB) to run both models simultaneously.

• Does MPS support explicit memory quotas or limits for each process when launched this way?
• Can i start my 2 Pods on this Node?
• If not, is there another recommended approach (e.g., MIG, CUDA_VISIBLE_DEVICES tricks, or GPU Operator configuration) to achieve similar memory partitioning on an ADA6000?

Thanks in advance for any guidance i am pretty new in this stuff

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. This issue will be closed in 30 days unless new comments are made or the stale label is removed. To skip these checks, apply the "lifecycle/frozen" label.

[ajavanma]

The stock GPU Operator/device-plugin MPS configuration cannot express an unequal 30 GB + 18 GB split. With two MPS replicas, the plugin sets each client's memory limit to half the GPU's reported memory—roughly 24 GB here. Two pods can each request one advertised shared GPU resource, but a client needing 30 GB would exceed that limit. The implementation calculates totalMemory / replicas.

MPS itself supports client memory limits. However, setting CUDA_MPS_PINNED_DEVICE_MEM_LIMIT inside a pod can only reduce the server's limit; it cannot raise the plugin's half-GPU limit to 30 GB. CUDA_VISIBLE_DEVICES controls GPU visibility, not memory partitioning. See NVIDIA's MPS environment-variable documentation.

Assuming you mean the RTX 6000 Ada Generation, that model does not support MIG; see the supported GPU list.

One option to evaluate is two time-slicing replicas, with a separately tuned --gpu-memory-utilization for each vLLM instance. This permits unequal application memory budgets, but provides no enforced memory isolation between pods. Both workloads must fit together, including runtime overhead and KV caches; 30 + 18 GB leaves no headroom on a nominal 48 GB card.

This is based on the current documentation and plugin implementation; I haven't tested your models on this GPU.