Skip to content

Repository files navigation

Metal GPU Telemetry – Real-Time GPU Profiling for Apple Silicon

I started this open-source project after realizing the need for a dedicated observability tool that can monitor real-time GPU resources and compute activity on Apple Silicon. Existing tools provided partial visibility, but none offered a lightweight, programmable way to capture per-kernel metrics, CPU↔GPU interactions, and system state in production workflows.

Metal GPU Telemetry was created to fill that gap—providing a clear, structured, and extensible way to understand how Metal compute pipelines behave at runtime.

Metal GPU Telemetry originated as an extension of Swift Telemetry, expanding its lightweight logging foundation into a complete real-time GPU observability system for Apple Silicon.

Where Swift Telemetry focuses on structured application-level diagnostics, Metal GPU Telemetry adds deep, kernel-level insight into Metal compute pipelines, delivering:

GPU timing and scheduling visibility

CPU → GPU interaction profiling

GPU memory footprint tracking

System load & thermal monitoring

Workload-aware performance context

Automatic performance hints and warnings

This makes it suitable for:

High-performance compute pipelines

3D mesh processing & graphics workloads

ML inference and GPU batch jobs

Metal shader debugging

Profiling performance regressions at scale

Features

GPU Metrics

GPU start/end timestamps

Kernel GPU time (ms)

Threadgroup execution width

Shared memory size

GPU queue depth

Hardware counters (macOS only)

Occupancy Metrics

SIMD utilization (%)

Threadgroup utilization (%)

Estimated active TGs per core

SIMD groups per TG

Kernel-level execution efficiency

CPU Metrics

CPU thread ID

CPU queue label

CPU callback delay

CPU encode→submit latency

System Metrics

CPU load (%)

Thermal state

Power source / constraints (AC/Battery)

Memory Metrics

GPU bytes allocated

System RAM

Recommended working set

Per-kernel footprint

Transfer Metrics

Bytes read

Bytes written

Future: Unified-memory fault counters

Workload Context

Each kernel execution links to metadata:

meshId

vertex/edge/face counts

batchId

stage name (e.g., compute_normals, reduce_descriptor)

Perfect for per-mesh performance analytics and regressions.

Hint System

Automatically detects performance issues:

High CPU→GPU latency

Low occupancy

Oversized threadgroups

Shared-memory underutilization

Memory pressure

Shows warnings cleanly in the JSON log.

JSON Telemetry Example

A typical telemetry packet looks like:

{ "timestamp": "2025-12-04T02:14:01.723Z", "kernelName": "compute_normals", "gpuTimeMs": 0.00787, "cpuToGpuDelayMs": 10.0823, "device": { "name": "Apple M3 Pro", "cores": 18, "tier": "pro", "family": "Pro" }, "occupancy": { "simdUtilizationPercent": 100, "threadgroupUtilizationPercent": 3.125 }, "cpu": { "cpuThreadId": 259, "cpuQueueLabel": "com.apple.main-thread", "cpuCallbackDelayMs": 11.12 }, "system": { "thermalState": "nominal", "cpuLoadPercent": 20.46, "powerConstraints": "ac_power" }, "transfer": { "bytesRead": 2, "bytesWritten": 2 }, "workload": { "meshId": "unknown", "meshVertexCount": 3, "meshFaceCount": 3, "meshEdgeCount": 3, "batchId": 0, "stage": "compute_normals" }, "hints": { "warnings": ["high_cpu_to_gpu_delay_vs_low_gpu_work"] } }

Grafana Dashboard

A full Grafana dashboard is included (import JSON provided), featuring:

GPU time timelines

CPU→GPU latency

Occupancy heatmaps

Memory usage

Per-mesh performance

Kernel distribution

Performance warnings panel

Supports:

Grafana + Loki

Grafana + InfluxDB

Prometheus exporters

Architecture Overview

Metal Command Buffer ↓ MetalGpuCounterSampler.attach() ↓ Collect CPU | GPU | System | Memory | Workload | Transfer ↓ GpuSampleV3 (full telemetry model) ↓ Recorder (JSON encoder) ↓ Logs → Loki / InfluxDB / ML Pipeline

Telemetry is captured asynchronously and incurs minimal overhead

Installation

Swift Package Manager .package(url: "https://github.com/LingaNPM/metal-gpu-telemetry.git", from: "1.0.0")

Then:

import MetalTelemetryCore import MetalTelemetryCounters

Usage

  1. Initialize sampler let sampler = MetalGpuCounterSampler(device: device, recorder: myRecorder)

  2. Wrap kernel dispatch sampler.attach( to: commandBuffer, kernelName: "compute_normals", pipelineInfo: info, threadgroups: 1, threadsPerThreadgroup: 32, workload: WorkloadContextV3( meshId: mesh.id, meshVertexCount: mesh.vertexCount, meshFaceCount: mesh.faceCount, meshEdgeCount: mesh.edgeCount, batchId: batchIndex, stage: "compute_normals" ), transferMetrics: transferMetrics )

Telemetry is emitted automatically when the command buffer completes.

Testing

Telemetry is validated by comparing:

encoder correctness

GPU timestamps

occupancy math

system metrics correctness

no missing fields (non-optional)

Run unit tests: swift run tests

Contributing

PRs welcome — especially improvements to:

system metrics

memory tracking

performance heuristics

visualization tooling

License

MIT License.

Support

If you need help integrating this with:

batch GPU workloads

ML pipelines

Metal-based engines

real-time dashboards

Open an issue or reach out.

xcrun -sdk macosx metal -c Shaders/*.metal -o Shaders/default.air
-target air64-apple-macosx xcrun -sdk macosx metallib Shaders/default.air -o Shaders/default.metallib

About

Metal GPU Telemetry is a observatory tool. Metal GPU Telemetry originated as an extension of Swift Telemetry, expanding its lightweight logging foundation into a complete real-time GPU observability system for Apple Silicon.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages