Search results

RuntimeWarning: coroutine 'process_before_invoke_callbacks' was never awaited

Status: Open.

#96 In awslabs/llmeter;

· acere opened on Jul 16, 2026

RunConfig.save() crashes with callbacks that have non-serializable state

Status: Open.

#94 In awslabs/llmeter;

· acere opened on Jul 3, 2026

Add a built-in profiling callback for per-invocation timing analysis

Status: Open.

#90 In awslabs/llmeter;

· acere opened on Jun 19, 2026

Plotting: no standard way to visualize percentage/rate metrics, and template is hardcoded

Status: Open.

#88 In awslabs/llmeter;

· acere opened on Jun 17, 2026

Add result filtering to generate datasets matching specific criteria

Status: Open.

#86 In awslabs/llmeter;

· acere opened on Jun 17, 2026

Feature: System metrics monitoring (CPU, memory, network I/O) during benchmark runs

Status: Open.

#83 In awslabs/llmeter;

· acere opened on Jun 12, 2026

OpenAI timeouts are broken, and Runner doesn't enforce them consistently

Status: Open.

#78 In awslabs/llmeter;

· athewsey opened on May 17, 2026

Unpinned s3fs pulls in an ancient and slow version

Status: Open.

#77 In awslabs/llmeter;

· athewsey opened on Apr 26, 2026

Estimate uncertainty of quantile/stats

enhancementNew feature or requestNew feature or request

Status: Open.

#59 In awslabs/llmeter;

· athewsey opened on Apr 6, 2026

Feature: Dynamic callback serialization mechanism

Status: Open.

#53 In awslabs/llmeter;

· acere opened on Mar 31, 2026

Plots/stats/analyses for scaling dynamics

enhancementNew feature or requestNew feature or request

Status: Open.

#33 In awslabs/llmeter;

· athewsey opened on Mar 23, 2026

Add automatic test as part of CI

enhancementNew feature or requestNew feature or request

Status: Open.

Task

#28 In awslabs/llmeter;

· acere opened on Feb 25, 2026