sunilmehta.si ↗   ·   LinkedIn ↗   ·   Building at ORGN   ·   Connect on X

I'm Sunil Mehta, a lead DevOps, MLOps and platform engineer and founding lead at ORGN. I work across cloud infrastructure, confidential computing, identity, and the systems that turn AI capabilities into dependable products.

My scope runs from terraform plan to production traffic: Kubernetes and GitOps, confidential compute, and secure delivery pipelines.

These highlights describe my professional experience. Most of the underlying work lives in private organization repositories.

Kubernetes Secure MLOps Pipeline — An MLOps pipeline on Kubernetes where every model that reaches production is reproducible, evaluated against the model it replaces, signed and verified at deploy time. Kubeflow Pipelines and MLflow train and gate; models ship as signed KServe images with SLSA provenance and an ML-BOM, and Kyverno refuses anything else at admission. One Helm chart is delivered to dev, staging and prod by Argo CD; live drift and accuracy alerts start retraining; a load drill holds the serving objectives. A real kind cluster runs 24 admission, network, monitoring, feedback, load and GitOps checks in CI on every push.

Explore the governance and threat model, serving and its drills, and environments and GitOps.

Kubernetes GPU Scheduler Benchmark — Which GPU scheduler should a platform run? Replays the same AI workload (training gangs, fine-tunes, inference, notebooks) through the default kube-scheduler, Kueue, Volcano and KAI Scheduler, first in a reference simulation and then on real schedulers in CI (kind + kwok GPU nodes), and measures utilisation, fragmentation, queue wait, fairness and cost, up to a 1,024-GPU fleet. Finding: out of the box only KAI bin-packs GPU pods by their GPUs; one setting fixes each of the others, and on the real Volcano scheduler it made the same trace 26% faster and 26% cheaper per useful GPU-hour. Every cluster version is pinned in one file, and a report says what upstream has released since and when it is old enough to adopt.

Explore the results, real-scheduler checks, and methodology.

Kubernetes Inference Routing Benchmark — How much does the router in front of LLM servers matter? Replays the same chat and RAG traffic through a Kubernetes Service and the llm-d Router (the Gateway API Inference Extension's endpoint picker) with load, prefix-cache and combined scoring, on kind clusters with simulated vLLM pods. Every router runs eight times, and only differences that held in every run are claimed. Finding: prefix-aware routing lifts the pods' cache hits from 64% to 80% and cuts median time to first token by 27%, but sends most requests to two of four pods, so load-aware routing keeps the best tail. Tuning the load scorer's weight or threshold did not spread the work; swapping in the router's active-request scorer did, beating prefix-plus-load on median, p95 and p99 time to first token in all eight runs, and with the prefix at weight 2 it keeps 83% cache hits with the best p95 of any router measured; on eight pods the prefix routers' pile-up doubles while it stays even, and on a control trace with nothing to cache it keeps its whole latency advantage, which shows the gain is the load scoring. Reading the pods' real cache contents from their KV-cache events, instead of the picker's estimate, finds two to three more points of cache hits in every run and a better tail, for a tokenizer round trip at the median. The scorers' own formulas, read from the router's source, explain each result, and a test checks every claim against the runs.

Explore the results, methodology, and the v0.3.0 release.

Agentic Code Review Tool — Scanner-grounded AI review for the whole pull request: 127 rules across app code, Kubernetes, Docker and Compose, Terraform, GitHub Actions, lock files and AI/agent code (OWASP LLM Top 10, MCP configs, agent skills and plugins, coding-agent settings), swept over Argo CD, Kubeflow Pipelines, KServe, MLflow and flytekit before each release so the false positives are fixed first, including the credential-stealing workflow shape of this year's GhostAction attacks, checkout tokens shipped inside workflow artifacts, RBAC escalation paths short of *, model archives extracted without a path filter, HTTP calls that wait forever, Go reviewed as fully as Python (SQL built with fmt.Sprintf, sh -c handed a built command, servers with no header timeout), and lock files that still trust the compromised emSign root, only on changed lines, with an optional Claude or local-LLM agent that verifies findings using read-only, repository-confined tools. Reports to GitHub, GitLab, SARIF and JUnit.

See a live AI review on a pull request, the rule catalogue, and the threat model.

AI DevSecOps CI/CD — A reference pipeline for shipping LLM and AI-agent services safely: prompt-injection tests mapped to the OWASP LLM Top 10, SBOM and AI-BOM generation, Trivy and secret scanning, keyless cosign signing with attestations, and Kyverno admission policies with an offline validator mirroring them in CI.

Explore the threat model, AI security test corpus, and workflows.

Jev Inference Router — An observable decision layer for local GPU inference: TypeSafe Jev chooses the handler, Python applies the policy, and local Qwen generates the answer, with visible probabilities, retrieval, Prometheus metrics, and reproducible evaluations against rule-based and classifier baselines.

Explore the architecture, evaluations, and operations guide.

LLM Inference Platform — GPU-backed LLM serving on Apple Silicon with vLLM-Metal, a Python streaming gateway, Prometheus/Grafana monitoring, reproducible benchmarks, and Kubernetes deployment templates.

Explore the architecture, operational tests, and benchmark evidence.

Infrastructure   Kubernetes · Terraform · Argo CD · Helm · GCP · AWS · DigitalOcean · Cloudflare
MLOps & serving   Kubeflow Pipelines · MLflow · KServe · vLLM
Systems & services   Go · Rust · TypeScript · Python · PostgreSQL · Redis
GPU scheduling   Kueue · Volcano · KAI Scheduler · kwok
Security   Intel TDX · OAuth 2.1 / OIDC · Kyverno · OPA · cosign · SLSA · OpenSSF model signing
Observability   Prometheus · Grafana · Loki · Tempo · OpenTelemetry · ClickHouse

• Make changes reviewable. Reproducible plans, GitOps reconciliation, and keyless delivery.
• Make failures understandable. Useful telemetry, explicit failure modes, and written runbooks.
• Make correctness repeatable. Regression tests for security fixes, billing paths, and migrations.
• Own the whole path. Design, delivery, operations, and the details between them.

Building AI platforms, developer tools, or secure infrastructure?
sunilmehta.si · Connect on LinkedIn · Connect on X · Email me