fix(ci): keep the transformers<5.0 pin through the package install

[he-yufeng]

The pin line in the CUDA path runs before pip install ., but pip keeps no constraints across invocations. The package install then re-resolves trl (floor transformers>=4.56.2) against framework.txt's <5.15.0 and jumps to transformers 5.14.1, after which the suite dies importing transformers.models.gemma3. Every PR is red on this right now (confirmed on mine and on #9854, same error in the same step).

Carrying the pin into the package install keeps the resolver on the 4.x line the suite is written for. Verified with the actual resolver: trl==0.29.1 + transformers<5.0 in one run selects transformers 4.57.6, which satisfies trl's floor and keeps gemma3 importable.

The NPU path already carries the pin inside its own command and is untouched. Whether framework.txt itself should drop <5.15.0 for users is a bigger support-range question I left alone here.

[RerankerGuo]

I traced the full failing run and found a second resolver mutation that this patch does not cover. In run 31179195838, the initial pin correctly installs Transformers 4.57.6. Later, tests/tuners/test_merged_linear.py loads damo/nlp_structbert_sentence-similarity_chinese-base with remote code; that model installs its own requirements (huggingface-hub==0.25.*, transformers==4.48.3) into the shared test environment. The log then shows Successfully installed ... transformers-4.48.3, and only afterward test_gemma3_template.py fails with ModuleNotFoundError: transformers.models.gemma3.

The same sequence appears in the five currently red PR runs (#9850, #9852, #9859, #9870, #9871). Therefore carrying transformers<5.0 into pip install . is useful for the initial install but will not make these runs green. The fix also needs to prevent/isolate the remote StructBERT requirements install, or restore the supported dependency set after that suite. Merely moving the Gemma import would mask the shared-environment corruption.

[he-yufeng]

@RerankerGuo Good catch, that StructBERT install explains the rest of the red runs. I went with the restore option in 5d534f7.

Preventing the model's requirements install would break test_merged_linear itself, since the remote code needs its pinned set to load. Isolating every suite in its own venv would work but is a much bigger change to the runner. Restoring after the suite keeps both sides intact: StructBERT runs on 4.48.3 inside its own suite, and the next suite starts on the supported set again.

What the patch does in tests/run.py:

• record_env_versions() snapshots the installed transformers/peft/huggingface-hub versions right after each env's requirements and dependencies are installed, i.e. the state the suites are written for (4.57.6 in the current image).
• restore_env_versions() runs when a suite finishes and reinstalls only what drifted. It is a no-op for suites that leave the environment alone.
• The hook sits in wait_for_free_worker where a completed worker is reaped, plus after each suite in the serial run_case_in_env path. With the CI default --parallel 1 the next suite only starts after the previous one is reaped, so the restore always lands between the polluting suite and the next one.

Your traced sequence is covered: test_merged_linear gets its 4.48.3 during its own run, then the restore puts 4.57.6 back before test_gemma3_template starts. huggingface-hub is in the snapshot list too, though reinstalling transformers 4.57.6 would pull it back above the required floor either way.

Verified locally with py_compile plus a standalone run of the record/restore logic (records versions, no-op without drift, reinstalls the pinned version with drift). I cannot reproduce the modelscope remote-code install on this machine, so the real check is this PR's CI run.

[he-yufeng]

Follow-up on the run of the restore commit: it still went red, and the log shows exactly why. The two errors are test_dataset failing to import (AssistantVocabTranslatorCache / QuantizedCacheConfig missing from the transformers on disk at that moment), i.e. it imported while the StructBERT 4.48.3 install was active.

The gap was in my first patch's coverage, not in the approach. With the CI default --parallel 1, all remaining suites of an env go into a single chunk that runs in one process (--suites <all of them>), so the pollution lands mid-process and the worker-reap restore between processes never gets a chance to run. d7c910d closes that: main() now runs each per-file sub-suite in turn and restores the pinned versions between them, which covers the chunk interior. Single-suite workers are untouched.

Verified locally: py_compile plus a unittest-semantics check that the gathered suite iterates per-file. The CI run on this head is the real proof again.