Problem: Currently, the synthetic workloads are overly simplistic (e.g., requests with a fixed input size and with constant KVCache reuse rate) and do not correspond to real use cases. Trace replay is an alternative, however, it would be useful to have a few parametrised synthetic workloads that model closer real use cases.
Desired functionality:
-
A RAG-like workload generator. Requests consist of a shared system prompt + a collection of documents + and a unique user request.
- The following knobs would allow customizing the workload: the length of the shared system prompt, the document universe (the total number of documents), the number of documents added to a prompt, the length of the user request, the length of the output. All lengths can be fixed or sampled from known distributions.
-
An synthetic Agentic workflow generator. The workload would consist of Agentic flows (trajectories), each flow consisting of multiple turns. At each turn, the input is constructed from the previous turn's input, output, and tool call tokens.
- Knobs: the input length of the start turn, the length of the output and tool calls, the total number of turns per workflow. All lengths can be fixed or sampled from known distributions.
Problem: Currently, the synthetic workloads are overly simplistic (e.g., requests with a fixed input size and with constant KVCache reuse rate) and do not correspond to real use cases. Trace replay is an alternative, however, it would be useful to have a few parametrised synthetic workloads that model closer real use cases.
Desired functionality:
A RAG-like workload generator. Requests consist of a shared system prompt + a collection of documents + and a unique user request.
An synthetic Agentic workflow generator. The workload would consist of Agentic flows (trajectories), each flow consisting of multiple turns. At each turn, the input is constructed from the previous turn's input, output, and tool call tokens.