clean cachyos-v4:latest build notes

[tiagocpontesp]

...sloppy slop slops...

I've just run uv pip install zstandard directly into the .venv inside the container, and resumed the build again from Step 4.

The build was failing at the rccl component due to a missing include in the net_tmp.cc (and net.cc) file for std::map. The error from the log was:
/opt/src/vllm/therock/build/comm-libs/rccl/build/hipify/src/transport/net_tmp.cc:240:8: error: no member named 'map' in namespace 'std'
I have fixed this by modifying both the original net.cc in the source repository (/home/t/works/BRAINS/therock/rocm-systems/projects/rccl/src/transport/net.cc) and the active build file to explicitly #include <map>, #include <string>, and #include <utility>.

The problem was that the CPython ./configure script was looking for llvm-profdata inside the custom TheRock LLVM compiler directory, where it didn't exist. 
Here is what I did to fix it:
1. Installed the system llvm package using pacman inside the vllm-builder container, which provides /usr/bin/llvm-profdata.
2. Patched /opt/src/vllm/build-vllm.sh to explicitly pass LLVM_PROFDATA="/usr/bin/llvm-profdata" directly to CPython's ./configure command.
3. Resumed the build script from Step 8.

some issue with:

Edit aotriton/third_party/triton/include/triton/Dialect/TritonGPU/Transforms/Passes.h
#ifndef TRITON_DIALECT_TRITONGPU_TRANSFORMS_PASSES_H_
#define TRITON_DIALECT_TRITONGPU_TRANSFORMS_PASSES_H_
#include "mlir/Pass/Pass.h"
#include "nvidia/include/Dialect/NVWS/IR/Dialect.h"
#include "triton/Dialect/TritonNvidiaGPU/IR/Dialect.h"
// #include "nvidia/include/Dialect/NVWS/IR/Dialect.h"
// #include "triton/Dialect/TritonNvidiaGPU/IR/Dialect.h"
namespace mlir {
namespace triton {

no clue what happened in between

  ━━ Backend Smoke Test Summary

  Backend              Result
  -------              ------
  vLLM                 ✓ PASS
  llama.cpp ROCm       ✓ PASS
  llama.cpp Vulkan     - SKIP
  Lemonade SDK         ✗ FAIL
  Ollama               - SKIP

Both are just re-running steps:
cd /opt/src/vllm
. vllm-env.sh
export RUSTUP_INIT_SKIP_PATH_CHECK=1
. "$HOME/.cargo/env"
# Fix Lemonade (reinstall PyPI package)
bash build-vllm.sh --step 36
# Fix Vulkan (rebuild llama.cpp with Vulkan + glslc now available)
bash build-vllm.sh --step 34

Fixed:
- Lemonade SDK ✓ — pip install lemonade-sdk (Step 36's "already installed" check was fooled by the git clone namespace package)
- Vulkan ✓ — pacman -S shaderc then re-ran Step 34 (glslc was missing)
Failing:
- When we installed lemonade-sdk from PyPI, it downgraded transformers from 4.57.6 → 4.53.2. vLLM requires transformers>=4.56.0. That's breaking vLLM.

"gated hf model used for testing" apparently

### "register fat binary failed" Errors
- **Frequency:** Continuous spam during AITer JIT compilation (~every few seconds).
- **Impact:** **Non-fatal** - build continues normally despite these errors.
- **Source:** Triton's `code_object.cpp` (PID changes, appears to be from Triton JIT registration).
- **Action:** Ignore. These are noise from Triton's code object registration, not compilation failures.
### Triton Build Fix
- **Problem:** Vendored Triton (`aotriton/third_party/triton`) downloaded pre-built LLVM with NVPTX dependencies.
- **Fix:** Set `LLVM_SYSPATH=/opt/src/vllm/local/lib/llvm` and `TRITON_OFFLINE_BUILD=1` to force use of locally built LLVM.
- **Patches applied to:** `aotriton/third_party/triton/setup.py` (dropped NVIDIA backend, only AMD).
### AITer JIT Compilation
- 67 total modules: 55 buildable for gfx1151, 12 CDNA-only (skipped).
- All 67 modules compiled successfully (62 built, 3 skipped CDNA-only during final run, plus 2 more skipped).
### Rust Toolchain Fix
- **Problem:** orjson build failed with `E0554: #![feature] may not be used on the stable release channel`.
- **Fix:** Installed Rust nightly toolchain: `rustup install nightly && rustup default nightly`.
## vLLM Fixes Applied (CRITICAL - both needed for gfx1151)
### Fix 1: `torch.fx.passes.split_module` `tuple_return` parameter (backends.py)
- **File:** `/opt/src/vllm/.venv/lib/python3.13/site-packages/vllm/compilation/backends.py`
- **Symlink also at:** `/opt/src/vllm/vllm/vllm/compilation/backends.py` (source tree)
- **Problem:** vLLM calls `split_module` with `tuple_return=True` based on `is_torch_equal_or_newer("2.12.0.dev")`, but the installed PyTorch 2.12.0a0+git9f8ad3e doesn't have this parameter in its actual function signature.
- **Fix:** Changed the version check to use `inspect.signature()` to detect whether `tuple_return` is actually present in the function parameters, rather than relying on version string matching.
- **Apply to both files:**
  1. `.venv/lib/python3.13/site-packages/vllm/compilation/backends.py`
  2. `vllm/vllm/compilation/backends.py` (source copy)
### Fix 2: `AttrsDescriptor.__repr__` produces invalid Python syntax (runtime/hints.py)
- **File:** `/opt/src/vllm/.venv/lib/python3.13/site-packages/torch/_inductor/runtime/hints.py`
- **Problem:** PyTorch Inductor generates Python code containing `repr(AttrsDescriptor)` where `AttrsDescriptor` is a pybind11 object whose `__repr__` returns `<triton.backends.compiler.AttrsDescriptor object at 0x...>`. This angle-bracket syntax is not valid Python literal syntax, causing `SyntaxError` when Inductor tries to import the generated code.
- **Fix:** Added `__repr__` method override to `AttrsDescriptor` class after import:
  ``python
  from triton.backends.compiler import AttrsDescriptor
  def __repr__(self):
      return f"AttrsDescriptor({self.property_values!r})"
  AttrsDescriptor.__repr__ = __repr__
  ``
- **Location in file:** Right after `from triton.backends.compiler import AttrsDescriptor` at line 45. y

resulting in

Simplified Status for Strix Halo (gfx1151)
✅ WORKS
Backend
llama.cpp ROCm
llama.cpp Vulkan
Lemonade SDK
⚠️ PARTIALLY WORKS (requires fixes)
Backend	Status
vLLM	⚠️ CODE GEN ERROR
❌ DOESN'T WORK (hardware/kernel issue)
Backend
vLLM with SmolLM2
vLLM with opt-125m
🔑 Key Finding
The aiter AMD ROCm kernels (compiled in Step 24) have a GPU memory page fault bug on gfx1151. This is a kernel-level issue, not something we can fix with Python patches.
What Models Work on vLLM
- ✅ Small models like opt-125m (uses vllm_c fallback kernels)
- ❌ Models that trigger aiter RMSNorm kernels
Bottom Line
For vLLM on Strix Halo: use small models or disable aiter. The large model support requires AMD kernel fixes upstream.